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ABSTRACT 

This report contains the comprehensive, written 
s 6l f — ss sessments completed by 10 Minnesota tech prep consortia. The 
1994-95 cohort of consortia was the second group to use the 
self-assessment. Within each tech prep consortium, the evaluation was 
completed by cross-district teams of teachers, administrators, and 
others who had been trained by the Minnesota Research and Development 
Center for Vocational Education. The summary is divided into four 
sections: cuj riculum and instruction, marketing, student assessment, 
and evaluation. Each section include's descriptions of the following 
areas: overall planning, staff development, special populations, 
curriculum integration, articulation, partnerships, evaluation, and 
retrospective. The report also contains the results of a 
brainstorming session in which participants shared successful and 
unsuccessful practices related to each section of tech prep. (KC) 
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Introduction 



This report summarizes the comprehensive, written self-assessments 
completed by ten Minnesota Tech Prep consortia during the 1994/95 school year. 
This cohort is the second group of consortia to go through the self-assessment 
process, the results of the first group are described in a separate report 
(Minnesota Research and Development Center, 1995). The self-assessment forms 
an integral part of the Minnesota Tech Prep Consortia Evaluation System that nas 
been designed by the Minnesota Research and Development Center for Vocational 
Education (MRDC). The MRDC is the independent, third-party evaluator of Tech 
Prep in Minnesota, under contract by Minnesota State Colleges and Universities. 
The self-assessment is completed by each consortium in its third year of 
implementation, two more cohorts are scheduled to follow in 1996 and 1997. 

Within each Tech Prep consortium, the self-assessment was completed by 
cross-district teams of teachers, administrators, and others who had been trained 
by the principal investigators of the MRDC. The self-assessment instrument 
consists of a number of specific questions addressing four Tech Prep Systems: 

• Curriculum and Instruction 

• Marketing 

• Student Assessment and Evaluation 

• Support Services and Counseling 

Each Tech Prep System contains seven Tech Prep System Activities: 

• Overall Planning, 

• Staff Development, 

• Special Populations 

• Curriculum Integration 

• Articulation 

• Partnerships 

• Evaluation 

Each of the four Tech Prep Systems also asked for retrospective thoughts 
about Tech Prep activities. For each cell of the resulting matrix of Systems and 
System Activities (see page 6), a consortium was asked to report on activities 
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relating to planning, implementation, and improvement, the three stages of the 
continuous improvement cycle. 

The completed self-assessments were returned to the MRDC, reviewed for 
completeness, and content-analyzed to identify common and unique activities 
within and across consortia. A summary of responses across consortia was 
presented to representatives of the cohort who reviewed it for accuracy. Section II 
of this report contains a validated summary of this analysis. 

Because the self-assessment system was designed to encourage continuous 
improvement, the review of the summary by cohort members was followed by a 
Peer Brainstorming Session conducted by the MRDC. The Tech Prep coordinators 
and members of the ten consortia, Tech Prep leadership at the State level, and 
MRDC staff participated in the session. The participants compared the status of 
Tech Prep activities within their own consortium with the aggregate findings of 
the Cohort Summary Report which served as a peer group benchmark. 
Participants then brainstormed and shared successful and unsuccessful 
practices related to each section of Tech Prep which had been evaluated. The 
results of this session are contained in Section III of this report. The consortium 
representatives also provided feedback to the MRDC on the self-assessment 
summary and reporting process. This information will be used to improve the 
format and process of analyzing and summarizing the self-assessment 
information. 

The content of the self-assessments reflected the variance among consortia 
in terms of size, amount of resources, experience with Tech Prep, and number of 
related or unrelated events that occurred during the reporting period. Despite the 
required effort of selecting and training teams, collecting information on a wide 
variety of Tech Prep related topics, and preparing a comprehensive written 
report, the self-assessment process was perceived as highly useful by many 
participants. The self-assessment process served as a focal point for reviewing all 
Tech Prep related activities from a system-wide perspective. 

As this report is published, the training for the next group, comprising six 
Minnesota Tech Prep consortia, is un lerway. Along with other initiatives, such 
as the frequent meetings of Tech Prep contact persons of all consortia in the State, 
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this report provides a means for sharing the expertise of this group of consortia, 
and for helping to guide the way for subsequent cohorts to build strong and 
successful Tech Prep programs throughout the State. For State-wide planning 
and assessment purposes, this report forms the first data point in a series of 
longitudinal, qualitative studies that will monitor the progress of Tech Prep 
implementation in Minnesota. 



The Minnesota Tech Prep Self-Evaluation System 

In the self-assessment, each consortium reported on the planning, 
implementation, and improvement activities related to Tech Prep for each of four 
Tech Prep Systems and seven Tech Prep System Activities. The resulting matrix 
is shown on the following page. Section II of this report summarizes the 
responses of the 1994/95 cohort across all ten consortia, and represents a 
summary profile or cohort benchmark. No single consortium is identified by 
name in this report. The page numbers printed in the matrix cells refer to 
Section II. 



Reference: 

Minnesota Research and Development Center, (1995). Minnesota tech prep 
consortia evaluation system: 1993/94 cohort self-assessment summary report . 
University of Minnesota, St. Paul, MN: author. 
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Retrospective: Overall Planning 



Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Retrospective - Overall Planning 



Planning 



What strategies worked best during the overall planning process for 
implementing Tech Prep in the schools throughout your consortium? 



Virtually all consortia emphasized the importance of broad stakeholder 
involvement to the success of planning efforts. The changing economic world 
heightens the need for business and labor involvement. One consortium sought to 
appoint a coordinator at each high school to facilitate involvement of all secondary 
and post-secondary faculty and staff. 

Organization was seen as important, with central leadership being supportive 
rather than directive. Regularly scheduled meetings helped to keep the process 
moving. One goal mentioned was to achieve early victories to develop an 
atmosphere of success. Pilot programs were valuable planning and training tools. 

Attitude was also reported as important. Having a strong vision, being flexible, 
listening, and being ready to innovate were cited as characteristics of successful 
planning teams. 

What pitfalls would you caution a new consortium about when planning 
to implement a Tech Prep initiative? 

Consortia frequently cited failure to set realistic goals, allow sufficient time, and 
assess school readiness for change as pitfalls. There needs to be recognition of the 
risk involved in change. 

Some cited organizational problems, Tech Prep being subject to unstable funding 
for example. Several stressed the need for a common vision of Tech Prep, and not 
to position Tech Prep as focused on low ability students. 
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Retrospective: Overall Planning 



Implementation 

What difficulties in implementing Tech Prep plans did you not 
anticipate and would revise if you had it to do over again? 



Several consortia stressed difficulties with involving various stakeholders, such 
as business and the community colleges. Another theme was the resistance to 
applied academics and the associated tendency to label applied courses as being 
for low ability students. Finally, many consortia discussed problems with time, 
citing the need for time to undertake widespread systemic change, the problems 
associated with districts or schools being at different points in implementation, 
and the time to do the work of Tech Prep. 

When implementing the total Tech Prep program, in what areas did you 
experience the least problems and why? (ex., curriculum, 
articulation, timelines?) 



Most consortia agreed that the articulation process proceeded well. The technical 
college staff and faculty are supportive of the process, and the task of articulation 
fosters better communication and networking between secondary and post- 
secondary faculties. Some reported that curriculum integration works well, 
especially with the mini-grant, pilot process. Employing standard curriculum 
materials, like CORD, is useful. One consortium singled out the consortium 
structure itself as useful. In this case the members had been successful with 
previous joint efforts. Several noted that exchanging visits with business and 
industry was a useful process. 

Continuous Improvement 

In which areas of activity during the implementation of Tech Prep did 
your consortium make the greatest improvements? Why? 



Most consortia singled out the articulation process as the most successful, with 
its attendant improvements in communication. Others mentioned course 
integration and the associated team teaching, the marketing process, and the 
general acceptance of the Tech Prep vision. One consortium reported great 
improvement in the career assessment process, an area most consortia 
mentioned as needing more improvement. 

In which areas of activity during the implementation of Tech Prep does 
your consortium need to make the most improvement in the future? 
Why? 

Many consortia listed the development and implementation of applied couues as 
needing mor3 improvement, as well as the career planning and assessment 
process. Sfveral cited specific areas within the general theme of school-to-work 
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Retrospective: Overall Planning 



processes, including apprenticeship programs, and worksite education. Some 
mentioned the related need to work more closely with business, industry, and 
labor. 

A second focus area for improvement was extending the Tech Prep concept to 
relationships with four-year colleges and the community colleges; it was also 
mentioned that there was need to do a better job of communicating to parents the 
value of two year technical career programs. 

Finally, one consortium reported the need for improvement of the assessment of 
student performance. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Overall Planning 



Planning 



Describe the mechanisms you planned for implementing the Tech Prep 
curriculum and instruction goals from your consortium’s original 
strategic plan (e.g., committees, planning groups)? 

Describe the overall goals, activity plans, and timelines with which you 
charged personnel assigned to develop the Tech Prep curriculum 
and instruction during the project. 



All consortia accomplished planning for Tech Prep curriculum and instruction 
through Steering or Executive Committees and, in many cases, a committee 
structure consisting of implementation committees, sub-committees, and task 
forces. Those committees typically represented a broad coalition of Tech Prep 
stakeholders, ranging from district-level administrators, teachers, and 
counselors to business and industry representatives and comimmity leaders. The 
committees typically conducted a series of meetings with local representatives to 
assess school level and district level needs and resources. There was an 
emphasis to incorporate the consortium-wide strategic plan to the extent possible 
while allowing enough flexibility at the local level. Two consortia sought 
guidance from Tech Prep implementation models in other states, namely Oregon, 
Massachusetts, and Kentucky. One consortium examined the European 
apprenticeship model and another sought to tie Tech Prep to a four-year youth 
apprenticeship program. One consortium used CORD materials for assessment 
and development of curricula. 

The ten consortia unanimously focused on the development and implementation 
of articulated, applied, and integrated curricula that were oriented toward 
providing applied academic skills and meeting current labor market needs. 

There was a strong emphasis on cooperation with the business community in 
designing courses, as well as on accountability through outcome-based 
approaches and the implementation of more rigorous and relevant learning. 
Several consortia mentioned the desire to reduce curricular overlap and to develop 
an effective information advisory system for students, parents, and the 
community. In general, the implementation time lines were left flexible to 
respond to the needs and time and resource constraints of local districts. 
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Tech Prep System: Curriculum and Instruction 



Implementation 

Describe how the mechanisms, goals, activity plans, and timelines for 
Tech Prep curriculum and instruction were put in place among the 
institutions within your consortium? 

The role of consortium leadership in driving implementation of curriculum and 
instruction at the district and school level ranged from a hands-off policy that left 
local districts and schools with a large degree of autonomy to tailor 
implementation to their specific needs and the resources available, to a more 
centralized role whereby the consortium actually prioritized the development and 
implementation of applied academic courses. In all cases, however, the 
consortium was instrumental in providing information about available resources, 
organizing workshops, providing an up-link to the Minnesota Department of 
Education, providing staff and curriculum development, and, in some cases, even 
providing grant opportunities and instructional resources. At the local level, 
there are, in most instances, contact people identified who carry responsibility for 
instructional and curriculum development. This task is carried out by teams or 
committees; one consortium reported having contracted with a number of 
teachers to lead the curriculum effort. The schools and districts used a variety of 
resources to drive curriculum development and instructional design: inservice 
training and education took place in some cases, as did attendance at regional, 
state, and national conferences. Some districts visited out-of-consortium schools, 
others invited nationally known speakers on the topic. Many districts made use of 
third-party materials, such as those available from CORD (Center for Occupation 
Research and Development), PAVTEC (Portland Area Vocational Education 
Consortium) Federal Model, or AIT (the Agency for Instructional Technology). 

By now, all consortia have a process established by which to prioritize curriculum 
development, and all have some courses which are articulated across levels. 

Tech Prep courses and curricula are either purchased, developed in-house, or 
consist of team- taught units or new units within existing courses. In general, the 
pace of curriculum and instruction implementation is dictated by the availability 
of dedicated resources in terms of staff, funding, and space. One consortium 
reported the development of a new Tech Prep lab, and plans to refurbish all 
existing high school labs into Tech Prep labs. 



Continuous Improvement 

How were you able to determine if the mechanisms, goals, activity plans 
and timelines set for the Tech Prep curriculum and instruction were 
reasonable; what data did you collect? 

How are you using this information to develop new or revised overall 
goals and strategies for implementing the consortium's Tech Prep 
curriculum and instruction? 

Data collection efforts ranged from informal "pulse-taking ", like feedback from 
meetings and "sense of the staff', to more formal measurement efforts, including 
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Tech Prep System: Curriculum and Instruction 



the number of courses implemented, staff trained, students enrolled, use of 
library materials, and so on. Mechanisms included observation, survey, student 
focus groups, and teacher surveys. 

Improvement efforts included modifying goals (focus on delivery of service), 
modifying data collection efforts, and improving instructor preparation through 
inservice and externships with business. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Staff Development 



Planning 

Describe the process your consortium used to determine staff 
development needs for addressing Tech Prep-related curriculum and 
instruction issues. 

Describe the staff development activities your consortium planned to 
address Tech Prep-related curriculum and instruction issues. • 



Most consortia viewed staff development for curriculum and instruction to be driven 
by the curriculum implementation process. While there were some special concerns, 
such as learning to use coursebuilder software, generally consortia intended to use 
implementation teams to develop curriculum and put it in place. These teams would 
learn by doing, and then share what they had learned as the process grew. Several 
consortia used a mini-grant-writing process to develop team plans and allocate 
resources to these pilot projects. Consortium steering committees could then observe 
these pilot projects and determine the staff development needs of the pilot teams, 
planning to meet the same needs in the wider implementation. A second process 
common to several consortia was to look to the steering committee or Tech Prep 
coordinator to identify the goals of Tech Prep within the consortium, and to clarify the 
values that would guide implementation. A common result was to recognize the need 
to provide training to familiarize the entire district staff with Tech Prep, not just Tech 
Prep staff. A third process was to plan to work with and build upon the work of staff 
development resources in place within the consortium, one example cited being the 
HRD Committee at the Technical College. 

A key activity planned in most consortia was the use of workshops/inservice. 

Some workshops would focus on developmental needs, like teaching styles; while 
others would focus on curriculum issues, like examining CORD materials. A 
second key activity, discussed further in 'Partnerships', is cross visitation of Tech 
Prep staff with representatives of business and industry. Lastly, most consortia 
encouraged staff to participate in regional and national workshops, in order to 
develop a 'big picture' of Tech Prep. 
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Tech Prep System; Curriculum and Instruction 



Implementation (What was actually done?) 

Describe the staff development activities your consortium conducted 

(provided) to implement Tech Prep curriculum and instruction during 
the project. 



Virtually all consortia actually conducted workshops as planned. One typical 
consortium structured workshops to: mix participants (for example, sometimes 
post-secondary and secondary staffs would meet together, at other times 
separately), rotate sites and times, use train the trainer approaches, and to have 
vendors lead workshops concerning commercial materials. Typical 
developmental workshops were led by academics with expertise in the content 
area. Many workshops focused on more technical issues, like examining the 
SCANS material for guidance in curriculum planning. Some consortia reported 
developing videos for use in workshops. 

Site visits by Tech Prep staff focused on visiting businesses, other Tech Prep sites, 
and other relevant developments like career academies. Frequently, consortia 
reported that business representatives in turn visited the schools. One consortium 
reported holding workshops concerning useful math and communication 
applications led by business and industry representatives. 

The Tech Prep coordinator's activities were seen as central to staff development. 
Coordinators stayed available to implementation teams and individual staff members 
to assist with problems. One consortium reported that the coordinator repeatedly 
visited new sites as they implemented new courses and curriculum. These visits 
contributed to staff morale and provided an opportunity to discuss specific issues (see 
Special Populations for example). 
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Continuous Improvement 

How were you able to determine if the Tech Prep curriculum and 

instruction staff development activities were successful; what data did 
you collect? 

How are you using this information to develop new or revise staff 

development activities for implementing Tech Prep curriculum and 
instruction? 



Most consortia reported a variety of informal data collection efforts aimed at 
answering whether their staff could implement Tech Prep curriculum and 
instruction. These included informal staff surveys, and observation of 
participation in meetings, based on the idea that the depth and quality of 
questions asked could be a gauge of understanding. Some sites tracked the 
requests for follow-up information after meetings, or the level of use of library 
materials. Feedback was sought from a variety of stakeholders (see Partnerships), 
including students, parents, and instructors. 

More quantitative approaches included tracking the number of applied courses, 
tiered applied courses, and enrollment levels and patterns in those courses. Most 
consortia reported collecting attendance and assessment sheets for each workshop. 
One consortium reported using follow-up information to move away from or improve 
upon self contained curriculum materials like CORDS and AID. Most reported the 
need to simply do more of what has been planned - more business visits, more 
inservice on applied curriculum, and more use of experienced staff to train, support, 
mentor, and coach others. Consortia generally reported that administrative support 
for Tech Prep activities is critical, and that guidance counselors need an updated 
understanding of Tech Prep. Many consortia reported identifying needs that have not 
yet been met, including coordination of Tech Prep with member schools, pursuit of 
system wide development, staff development for new people joining in the middle of 
the project, and better use of performance based assessment. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Special Populations 



Plannin g 



Describe how your consortium planned curriculum and instruction 
strategies for the inclusion of special populations in your Tech Prep 
programs. 



Describe the curriculum and instruction approaches your consortium 
selected to provide for the inclusion of special populations in your Tech 
Prep programs. 



One consortium reported involving a pre-existing Transitions Committee from 
each school to coordinate planning, while another planned to include 
representatives from special populations in the planning process (teen parents, 
for example). Many consortia plan to use IEPs for tracking and follow-up, and to 
provide special inservice for counselors. Others reported an identified need to use 
a variety of instructional delivery systems to meet the needs of special populations 
Some of those using mini-grants required consideration of special populations in 
mini-grant proposals. 



Implementation (What was actually done?) 

Describe how the selected curriculum and instruction approaches for the 
inclusion of special populations in Tech Prep programs have been 
implemented throughout your consortium. 



Consortia reported that they have targeted special populations for more attention 
from guidance staff, hired non-traditional staff (e.g. females in industrial 
technology), implemented interdisciplinary teaching (e.g. special ed. and 
mainstream teachers), reviewed courses for equity, tested for and delivered 
developmental math instruction, and made special populations a point of 
discussion during coordinator visits. 
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Tech Prep System: Curriculum and Instruction 



Many consortia followed the idea that hands-on instruction works well for special 
needs students, but are concerned not to label Tech Prep as special education. 
Rather, all students should be included in Tech Prep opportunities, and course 
integrity must be maintained. 



Continuous Improvement 

How were you able to determine if the selected curriculum and instruction 
approaches led to the inclusion of special populations in your Tech Prep 
programs? What data did you collect? 

How are you using this information to improve curriculum and instruction 
for the inclusion of special populations in your Tech Prep programs? 



Consortia reported monitoring IEPs, the level of special population enrollment in 
technical college cluster courses, the number of students in non-traditional areas, 
and improvements in the grade point average. 

Some directly monitored success of special populations in terms of number 
enrolled, using as sources the HS follow-up, Perkins application, and MNCIS. 

Areas identified for improvement included the need to modify courses, and the need 
to improve coordination with the technical college equity coordinator. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Curriculum Integration 



Planning 



Describe the planning process your consortium used to address the need for 
integration of academic and vocational course content in the Tech Prep 
programs. 



Describe the methods and procedures your consortium selected for 

integrating academic and vocational content in Tech Prep curriculum 
and instruction. 

One consortium conceived curriculum integration to happen in levels: projects, 
units, courses, and so on. There was concern expressed that courses go 
somewhere, that is lead to a degree. Consortia planned to identify core curricula 
in clusters. One planned to start with apprenticeship curriculum. One 
consortium saw integration as driven by secondary /post-secondary cooperation, 
while others focused on service learning, applied learning, work readiness, use of 
standards like SCANS and the MN graduation rule, and applied academics in 9th 
and 10th grades. One consortium identified planning for curriculum integration 
as the Tech Prep coordinator's task. 

Generally, piloting and replication were key strategies. This approach avoids 
large scale mistakes, and lets teachers become resources. Other strategies 
included using mini-grants, using technical college instructors as advisors, and 
teaming of vocational and academic instructors. 



Implementation (What was actually done?) 

Describe how curriculum integration was implemented in the schools 
throughout your consortium. 



Implementing curriculum integration was often hand-in-glove with related 
methods like team teaching. One consortium found the ALCs were adapted to 
integration since the schedule and curriculum were more flexible. Depending on 
factors like staff readiness, the depth and breadth of the integration progress 
varied across schools within < he consortia. Consortia reported implementing 
applied curricula in all schools in areas like math, commun ations, principles of 
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technology, biochemistry, work readiness, and communications. A goal was to 
identify skills students need to learn at one level in order to succeed at the next 
level. 

A key method was team teaching, using partnerships like math with industrial 
technology, physics with technology, English with keyboarding, and English with 
technology. One consortium reported requiring all 9th graders to take one course in 
future technology. In many cases school-level implementation controlled timing. 
Consortia reported developing Tech Prep magnets, one reported having plans for an 
entire Tech Prep school. Tech Prep was often related to four year apprenticeship and 
other school-to-work transition programs. 



Continuous Improvement 

How were you able to determine if the curriculum integration approaches 
you chose were appropriate for the integration of academic and 
vocational content in your Tech Prep programs? What data did you 
collect? 

How are you using this information to improve curriculum integration in 
your Tech Prep programs? 



Consortia monitored student requests for tiering, the number of courses 
integrated, the level of enrollment, and student achievement. Informal feedback 
included improved communication between students, parents, and instructors. 
Some recognized the need to start measuring post secondary aspects. 

Information about success stories was shared within one consortium to promote 
interest and enthusiasm for Tech Prep in general and curriculum integration in 
particular. There was some discussion of innovations like career academies. 
Recognized areas for improvement included ways to overcome barriers to team 
teaching, and inservice on alternative assessment. Generally, the barriers reported 
were time and money and the need reported was to expand what consortia were 
currently doing. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Articulation 



Planning 

Describe the process used to plan curriculum articulation across secondary 
and post-secondary institutions in your consortium? 

Describe the articulation strategy your consortium selected to address 
curriculum articulation within the Tech Prep programs. 

Consortia plans included developing a simplified articulation form, and focusing 
on teacher level agreements. Consortia planned to pre-set a level of agreement 
between both content and outcomes in order for courses to be candidates for 
articulation, and some expected that the college would train the secondary level 
on how to meet the rules (80% overlap, for example). Some planned to use more 
than one high school course to articulate with one college course. 

A common plan was to have all agreements include a sunset provision. Some 
consortia planned for learner outcomes and authentic assessment in articulation 
agreements. Some planned to rely on teamwork in integration efforts to foster 
articulation. In some cases secondary and post- secondary faculty jointly reviewed 
each other s curriculum. Some developed a matrix of course requirements by 
cluster. 



Implementation (What was actually done?) 

Describe how Tech Prep curriculum was articulated across secondary and 
post-secondary institutions in your consortium. 

Consortia reported that faculty were able to discover agreement or modify 
objectives in order to reach agreement. Some reported vertical and horizontal 
articulation within a technical college and among colleges, including four year 
institutions, such as Bemidji State, UM-Crookston, and Moorhead State. 

Consortia reported developing advanced standing agreements, and inter-college 
agreements among the technical colleges, evidence that Tech Prep was having an 
impact on the post secondary level, including community colleges and private 
technical colleges. One consortium reported that a technical college would do 
introductory courses for delivery by ITV. 
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Materials reviewed as part of the articulation process included course descriptions, 
goals, and texts and references. Outcomes included certificates for advanced 
placement or credit, and the identification of prerequisites. Some had developed a 
checklist to guide the articulation process. 

Continuous Improvement 

How were you able to determine if 'urriculum articulation between 
secondary and post-secondary institutions was successfully 
implemented in the schools in your consortium? What data did you 
collect? 

How are you using that information to improve curriculum articulation 
in your Tech Prep programs? 



Data collection included the number of articulation agreements; the level of 
enrollment, the number of certificates, and Tech Prep certificate growth. One 
reported a plan to gather baseline data on post secondary enrollments, others 
planned to survey instructors guided by key measures from the strategic plan, 
and to gather e mployer and student reports on curriculum effectiveness. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment S ummar y Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Partnerships 



Planning 

Describe the planning process you used to insure the participation of the full 
range of stakeholders in the design of Tech Prep curriculum and 
instruction. 

Describe the major components of your designed plan to ensure that 
important stakeholder groups were involved in the selection or 
development of Tech Prep curricula. 



A common theme was recognition that there may be different stakeholders at 
each of the consortium, district, and school levels. Consortia reported that 
partners were given special projects and assignments to focus their participation, 
secondary and post secondary students were included, teacher engaged in 
externships, and the leadership emphasized the value of partnerships. Planning 
focused on ways to involve local forces: agriculture, business and industry, labor, 
parents , students, teachers, and administrators. One consortium reported 
reciprocal bartering with some stakeholders (materials for training). Some tied 
workplace learning to Tech Prep. 



Implementation (What was actually done?) 

Describe how stakeholder groups were involved in designing or selecting 
Tech Prep curricula in the schools in your consortium. 

One consortium reported surveying employers concerning SCANS requirements, 
not just taking them as is. Several consortia reported using SCANS to identify 
skills, and using partnership groups to designate target clusters. A frequent 
practice was to have teachers meet with local businesses. 

Consortia leadership held discussions with special population representatives, 
parents, and student focus groups to design curriculum implementation. 
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Continuous Improvement 

How were you able to determine whether the participation of stakeholders in 
the Tech Prep curriculum design process was successful? What data did 
you collect? 

How are you using this information to improve the participation of 
stakeholders in the future revision of Tech Prep curricula? 



Consortia reported that partners took ownership and are interested in more 
classroom involvement and mentoring. Most reported a sense of broad based 
support. Data collection included minutes of partnership meetings. 

Some specific changes were to alter the time of day for meetings to meet the 
scheduling needs of business members and others, and changing types of 
implementation processes. One consortium found that they should improve work 
based training, another felt they should form a Business Advisory Group as a 
subcommittee of the Local Quality Council. One consortium reported it was hard 
to get participation because of geography and the pace of changes, so they could 
only expect sporadic involvement. 

Areas for improvement reported include the need for data concerning the actual 
value of Tech Prep, and the need to disseminate more information to parents. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Curriculum & Instruction 
System Activity - Evaluation 



Planning 

Describe the process your consortium used to plan for the assessment of 
Tech Prep curriculum and instruction? 

Describe the processes developed by your consortium to evaluate the 
adequacy and appropriateness of the Tech Prep curriculum and 
instruction. 

One consortium hired a consultant, and developed working definitions of applied 
learning and a global view of integration against which to measure outcomes 
(based on the work of Dr. Willard Daggett). Other consortia reported linking Tech 
Prep assessment to other plans, for example to North Central accreditation, or 
district strategic plans. 

One consortium planned for assessment to be coordinated by the technical college. 
A concern was to make all instruments user friendly. One suggested that 
identifying successful completers would be the best indicator. Another suggested 
that assessment be tied to technical college performance and graduation. 



Implementation (What was actually done?) 

Describe how these assessment strategies were implemented in the schools 
throughout your consortium? 



Very little detail was reported concerning actual assessment activities. Data 
collection included pre and post interviews of students, and the use of standardized 
data collection like the HS follow-up. 
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Continuous Improvement 

How were you able to determine whether the curriculum and instructional 
assessment approaches you implemented were successful? What data 
did you collect? 

How are you using this information to improve the assessment of Tech Prep 
curriculum and instruction throughout your consortium? 



Consortia reported using student surveys and focus groups, one secondary result 
being that students are pleased to be asked. Other data collection included survey 
of teachers and survey of administrators. Data used include number enrolled, 
courses maintained, credit granted, student success, and student placement. One 
consortium said that its role was to set the model, and to leave data collection to 
the school level. Some consortia reported data collection to include the number of 
agreements, number of AP credits, and the degree of adoption of applied 
academics. 

Some consortia reported as a consequence of assessment the effort to move away 
from, or to redefine and refine standard curricula, e.g. CORD or AID. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment S ummar y Report 



Retrospective - Curriculum & Instruction 



Planning 

What planning processes worked best for your consortium during the 
planning phase of designing Tech Prep curriculum and instruction? 

Best planning practices included attending national meetings, coordinator visits, 
informal work with stakeholders, early stakeholder involvement, setting clear goals, 
having a vision of Tech Prep, getting public support early, and getting inside support. 



Based on your experiences, what pitfalls would you caution a new 
consortium about when planning Tech Prep curriculum and 
instruction? 

Consortia recommended not to push the vendor materials, and to get more than just 
interest from stakeholders. One suggested the need to recognize that it was not 
automatic that teachers were actually able to deliver instruction in this environment. 
There was a need for patience. Barriers included the lack of parental interest, and a 
lack of resources. 



Implementation 

When implementing Tech Prep curriculum and instruction, in what areas 
of activity did you experience the greatest successes (ex., coordination, 
articulation, timelines)? Explain why you think this occurred. 



Consortia reported that articulation brought mutual respect and was good 
marketing. Integration needed time and money, and it was important to establish 
the need and desire for the program. One consortium reported that an applied 
workshop for teachers held at the technical college was a model for the program 
itself. Consortia reported the need for inclusive curriculum, and for funding of 
cornerstone courses. It was important to involve parents and to do business visits 
and other forms of school-community communication. In the classroom, students 
found successes in a hands-on environment. One consortium counseled to ignore 
the naysayers, and to focus on those ready and willing to do the work. Many 
recommended to do articulation first. It was the least expensive, there was the 
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least resistance, there was is student acceptance, and it was is a good way to 
establish networks between secondary and post-secondary levels. 



Please describe the three of four greatest difficulties you have encountered in 
setting up curriculum and instruction for Tech Prep programs? 

There was a need for improved career counseling. Consortia reported the need to 
educate the public that not everyone needs a college degree. There was uniform 
agreement on the lack of time and money. There was a need for improved teacher 
preparation. One reported that other programs got in the way . There was 
concern over the idea of tracking. There were turf battles, and a lack of acceptance 
by academic staff. Some consortia reported turnover on committees creating 
difficulties with continuity of effort. Again, it was easier to focus on articulation 
than on integration. 



Continuous Improvement 

In which areas of activity during the implementation of Tech Prep 
curriculum and instruction did you experience the greatest 
improvement? Why? 

Consortia reported that hands-on learning excited students, that employer 
involvement in curriculum issues was useful, and that the assessment process 
was getting deeper and more profound as its value became clearer. Teachers had 
the most success with team teaching, and hands-on instruction. 



Describe the areas that were the most difficult to improve in the 
development of Tech Prep curriculum and instruction in your 
consortium and explain why you think they were? 

When Tech Prep courses were taken by 9th & 10th graders, they could not get AP 
credit. Consortia reported in as many ways as possible that time and money are 
key constraints. There was a need for money for applied materials. Geographic 
coordination was difficult in rural areas. It was also difficult for small 
communities to get business involvement on teams, and to provide experiential 
education, opportunities. Some consortia felt they were too large. 

Teachers needed common prep time to work on integrated curriculum and team 
teaching. There was some difficulty implementing cooperative learning, some 
students had trouble with teamwork. There was a problem with the acceptance of 
integration by staff, with issues like turf, a concern over the potential for watered- 
down curricula, a concern about ending up with only low-ability students, and 
concern over students who were not used to the challenge. Scheduling of work 
based components could be troublesome. There was concern over a perceived 
lesser value of a two- year degree, and that Tech Prep not get labeled as just for 
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low ability students. There was a lack of career development curricula. There was 
a lack of statewide uniformity on articulation with MN graduation rules, no clear 
definition of Tech Prep, and the required reporting system took too long. The 
entire program was seen as very dependent on funding. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Overall Planning 



Planning 



Describe the mechanisms you planned for implementing the Tech Prep 
marketing goals from your consortium’s original strategic plan (e.g., 
committees, planning groups)? 

Describe the overall goals, activity plans, and timelines with which 
you charged personnel assigned to develop Tech Prep marketing 
during the project. 

All consortia accomplished planning for marketing Tech Prep through a steering 
committee or similar team structure at the consortium level. This committee 
developed the vision, implementation timelines, and general marketing plans for 
the consortium, but typically left the individual district or schools ample room to 
modify and tailor the plan to their individual needs. In the majority of cases, the 
marketing plan was developed in cooperation with the districts, business and 
industry associations, such as the Chamber of Commerce, and the community at 
large. Some used available federal models, such as PAVTEC, for guidance. In 
general, the goal of the marketing plan was to inform the larger community of the 
value of Tech Prep and its opportunities for students, to present Tech Prep as a 
viable alternative to traditional four-year degree programs, to disseminate 
information about Tech Prep, and to recruit qualified students into the program. 
One consortium set the time line for implementing a viable marketing strategy at 
three years. 



Implementation 

Describe how the goals, activity plans, and timelines for marketing Tech 
Prep were coordinated among the institutions within your 
consortium? 

Consortia implemented their marketing plans in a fairly uniform fashion. All 
consortia developed and distributed a variety of promotional materials, such as 
brochures, newsletters, program guides, folders, pencils, banners, and 
individual Tech Prep logos. They also made use of the mass media through 
advertisements and articles in local newspapers and radio stations. Several 
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consortia produced a video, and one organized a Tech Prep Resource Library. 
Marketing to the public was done typically through presentations, career days, 
and open houses. Internally, there were presentations to students, inservice 
training for counselors and teachers, consortium-wide sessions and meetings, 
and exchange with members from other consortia. A particular emphasis was 
put in placing Tech Prep in the context of a changing world of work and jobs of the 
future that will require a different skill set. One consortium coordinated its 
marketing with the local technical college, another conducted student focus 
groups to communicate the idea of Tech Prep and generate ideas for marketing 
and promotion. 



Continuous Improvement 

How were you able to determine if the mechanisms, goals, activity plans 
and timelines set for the Tech Prep marketing were reasonable; what 
data did you collect? 



How are you using this information to develop new or revised overall 
goals and strategies for implementing the consortium's Tech Prep 
marketing? 

Most consortia relied on informally attained feedback as the primary means of 
determining the need for revising marketing materials or ascertaining the 
impact of their marketing implementation. This feedback came from a variety of 
sources: peers, staff, students, business and industry, and the community. One 
consortium used a formal survey of inservice teachers and another used focus 
groups to obtain feedback. Several consortia mentioned the state- wide data 
gathering instruments, such as the Consortia Self-assessment process and the 
Tech Prep Identifier Form as important input. One consortium formally tracked 
program participation data, the number of new programs, and the degree of 
financial support obtained, and used this information to assess the effectiveness of 
its marketing effort. The feedback was used primarily to revise marketing 
materials and to assess the effectiveness of the marketing strategy. Other 
consortia mentioned the benefit of sharing marketing materials and strategies 
with other consortia. Several consortia addressed the need for marketing to be 
continuous and ongoing, with input from a broad array of stakeholders, and the 
need to disseminate marketing information to a broad audience in order to obtain 
feedback and new ideas. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment S ummar y Report 



Tech Prep System - Marketing 
System Activity - Staff Development 



Planning 



Describe the process your consortium used to determine staff 
development and training needs for marketing Tech Prep in your 
consortium’s schools. 

Describe the staff development activities your consortium planned to 
deliver staff development for marketing Tech Prep programs. 

Planning for staff development in marketing Tech Prep was done at the local or 
district level in all but two consortia which adopted a centralized approach and 
developed a system-wide staff development plan. Only one consortium used a formal 
needs assessment method to determine staff development needs, the others used 
informal methods such as meetings with staff or planning meetings to become 
cognizant of student demand and staff interests and needs. The developmental 
activities were, in all cases, oriented toward the consortium-wide marketing plan (or 
the district-specific version thereof) with the main goal of enabling staff to 
successfully market Tech Prep to students, parents, business and industry, and the 
community at large. Among the activities planned were workshops, attendance at 
conferences, the publication of printed materials, as well as regular updates about 
developments in Tech Prep at the consortium, state, and national levels. 



Implementation (What was actually done?) 

Describe the staff development and training activities your consortium 

has conducted to address the Tech Prep marketing strategies. 

Staff development and training activities consisted primarily of inservice activities 
conducted by consortium representatives, workshops, attendance at regional, state, 
or national conferences, and site visits to other consortia. Information about Tech 
Prep was also regularly disseminated at staff meetings. A variety of printed 
materials supported the developmental activities. Many consortia reported that 
articles about the status of Tech Prep were regular features in school newsletters, 
and at least one consortium has designed and printed information booklet regarding 
Tech Pr«T> but one consortium involved teachers and staff actively in marketing 



MRDC, University of Minnesota © 1996 

O 

ERLC 



3 V 



Page 35 



Tech Prep System: Marketing 



Tech Prep. The exception was one consortium that had a marketing specialist on staff 
who conducted most of the marketing activities for that consortium. 



Continuous Improvement 

How were you able to determine if the staff development activities your 
consortium provided were successful in developing expertise needed to 
address your marketing needs? What data did you collect? 



How are you using this information to develop new or revised staff 
development activities to improve the marketing of Tech Prep programs? 



Most consortia reported that marketing Tech Prep was still in its initial stages, 
but recognized the need for continuous and ongoing efforts in promoting Tech 
Prep to all constituents. Only a few consortia collected data about their marketing 
effort. Most reported relying on informal means, such as verbal feedback or word- 
of-mouth. The information that was collected centered around feedback on the 
materials distributed and other information about future needs and areas of 
opportunity for Tech Prep. Such feedback included regional surveys and 
questionnaires regarding the level of knowledge and awareness of Tech Prep, the 
number of articulation agreements in place, and the types and amounts of 
promotional materials ordered. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Special Populations 



Planning 



Describe the process your consortium used to develop a Tech Prep 
marketing plan specifically for the inclusion of students with special 
needs. 



Describe the marketing activities your consortium planned to market 
Tech Prep programs to students from special populations and their 
parents. 

Planning for marketing of Tech Prep to special populations was conducted primarily 
in an informal fashion. Some consortia reported that there no formal plans for 
marketing to special populations, others mentioned their intent to be inclusive, 
targeting all students. Only a few consortia had actual representation and 
participation of special needs staff or special population advocates during the 
planning phase. Most consortia left it to the individual schools to develop plans to 
include special populations in their marketing efforts. 



Implementation (What was actually done?) 

Describe the activities your consortium undertook to specifically attract 
students from special populations to the Tech Prep programs in your 
consortium’s schools. 

Despite the lack of systematic and inclusive planning for recruiting special 
populations to participate in Tech Prep, all consortia reported conducting a 
number of activities targeted toward encouraging members of special populations 
to consider enrolling in Tech Prep courses. These activities included conducting 
workshops and inservices for counselors, other special needs staff, and parents; 
developing and distributing printed materials regarding Tech Prep to members of 
special populations; and holding events such as career days for non-traditional 
students. Two consortia included activities designed to encourage female 
students to participate in Tech Prep and achieve gender equity. One consortium 
reported an effort to coordinate other special needs grants with the Tech Prep 
grant. The overall stated goal for most consortia was to create inclusive programs 
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and encourage members of special populations to enroll in Tech Prep related 
courses. To this end, several consortia reported an effort to develop and 
implement materials that addressed both mainstream and special populations. 



Continuous Improvement 

How were you able to determine if the marketing activities your 
consortium conducted actually attracted students from special 
populations to your Tech Prep programs? What data did you collect? 



How are you using this information to develop new or revised marketing 
activities to attract students from special populations to your Tech Prep 
programs? 

While several consortia reported no effort to track program participation by special 
populations, most consortia are keeping count of the number of members of special 
populations enrolled in Tech Prep programs and are finding increasing enrollments 
by this group of students. Several consortia stated as their goal to continue attracting 
special needs students to Tech Prep and to present it as a viable and valuable 
alternative to traditional programs. One consortium reported the need to further 
modify and adapt its marketing materials and activities to the need of special needs 
students. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Curriculum Integration 



Planning 



Describe the process your consortium used to plan the marketing of 
curriculum integration in the schools of your consortium. 

Describe the activities your consortium planned to promote curriculum 
integration in the Tech Prep programs. 



In most consortia the planning process for marketing curriculum integration was 
conducted jointly by Consortium Leadership, local districts and schools, and 
involvement by business and industry. The prdcess typically involved determining 
the needs of schools, assessing the level of resources available, and gathering 
information about detail requirements of the proposed or planned integration effort. 
To this end, some consortia conducted visits to business and industry and consulted 
other consortia and districts. One consortium sponsored dinner meetings for high 
school and technical college staff, others organized workshops and joint planning 
meetings with different constituents. By and large, there was an effort to leave as 
much autonomy as possible to the local district while maintaining overall direction at 
the consortium level. 
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Implementation (What was actually done?) 

Describe how your consortium marketed curriculum integration to all 

groups both in and out of the schools in your consortium. 

Both the curriculum integration plan and ongoing and completed integration efforts 
were communicated to all parties involved through a variety of media: in 
presentations, workshops, conferences, inservice training, and through brochures, 
newsletters, and other written materials. In all cases, the intent was to educate and 
communicate curriculum integration to as wide an audience as possible. This 
included teachers, counselors, students, parents, industry and business partners, 
consortium leadership, and the community at large. Several schools sponsored 
summer writing projects, others made mini grants available to schools for the 
development of marketing materials that then would be shared with other schools 
and districts. One school identified specific individuals who were charged with 
promoting Tech Prep curriculum integration. A number of consortia reported that 
member schools sponsored events such as career nights and presentations by 
industry representatives to market Tech Prep. Frequently, districts developed 
materials available at registration to encourage students to consider integrated 
courses. In order to keep information current and accurate, one consortium reported 
that it held workshops for school staff regularly. 



Continuous Improvement 

How were you able to determine if the marketing activities your 
consortium conducted to promote curriculum integration in your Tech 
Prep programs were successful? What data did you collect? 



How are you using this information to develop new or revised marketing 
activities to improve curriculum integration in your Tech Prep 
programs? 



Most consortia assessed the effectiveness of their efforts to market curriculum 
integration along with other Tech Prep results indicators. They tracked the number 
of articulated courses and the number of students enrolling in those courses. Most 
consortia reported encouraging results that point to increasing enrollment and 
interest in articulated, applied academic courses, as well as an increased interest in 
advanced levels. The marketing effort has led to innovative approaches such as team 
teaching, and plans to integrate Tech Prep with Youth Apprenticeship or Career 
Academies. Other consortia cited an increase in the number of requests for 
information about Tech Prep from teachers and parents, and increased demand for 
promotional materials, as evidence of successful marketing. 

This positive information is used to reaffirm the direction and effort in marketing 
Tech Prep. Only one consortium offered insight into areas for improvement. Their 
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marketing activities are seen as needing to be redefined to involve regular academic 
teachers to a larger degree. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Articulation 



Planning 



Describe the process your consortium used to plan for promoting the 
articulation of secondary and post- secondary Tech Prep programs. 



Describe the activities your consortium planned to market articulation 
within its Tech Prep programs. 



Articulation and the promotion of articulation efforts went, in most cases, hand-in- 
hand, and was initiated in most cases by consortium leadership or committees 
appointed by consortium leadership. Through regular meetings, the appointed body 
developed a marketing strategy that included communication to all stakeholders 
involved, development of written and visual materials, and regular updates on the 
progress of articulation. In several instances, promotion was addressed jointly by 
several schools and districts. The appointed committee reported to the consortium 
leadership regularly. Only one consortium reported that articulation and the 
promotion of articulation was not one of its foremost concerns. 



Implementation (What was actually done?) 

Describe how your consortium marketed articulation of secondary and 
post- secondary Tech Prep programs to all groups both in and out of the 
schools in your consortium. 



The promotion of articulation for Tech Prep progressed along with the articulation 
efforts and typically included orientation sessions for faculty, counselors, students, 
parents, and the community at large; written materials, such as brochures, flyers, 
newsletters, and booklets; and, in many cases, videos produced by the consortium. 
Many consortia mentioned that information about articulated courses was regularly 
included in registration booklets. Many consortia also displayed the progress of 
articulation efforts visually in the form of flowcharts or graphs that explained the 
available articulated programs and courses. Several consortia publicized articulation 
during career days or similar public events, and some held joint events with the 
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respective technical college. Graduation ceremonies and formal certificate award 
festivities were also often used to publicly recognize and promote Tech Prep. Many 
consortia mentioned that they kept teachers and counselors regularly informed of 
new articulation agreements, so that they, in turn, could pass along the message and 
promote Tech Prep through word-of-mouth. Lastly, a number of consortia involved 
the public media to celebrate and promote the completion of articulated courses or 
programs. 



Continuous Improvement 

How were you able to determine if the marketing activities your 
consortium conducted to improve the articulation of your Tech Prep 
programs were successful? What data did you collect? 

How are you using this information to develop new or revised marketing 
activities to improve articulation within your Tech Prep programs? 



The most frequently mentioned method to assess the success of articulation was to 
track the number of articulation agreements and the numbers of students 
enrolled in or successfully completing articulated courses. This information is 
being recorded at the district and at the consortium level. Other consortia record 
the number of certificates issues upon successful completion of an articulated 
course or series of courses. This information, along with informal feedback from 
teachers, counselors, students, parents, and the community, is ned to gauge the 
effectiveness of the marketing effort and to fine-tune or adjust the marketing 
strategy for Tech Prep. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Partnerships 



Planning 

Describe the process your consortium developed to ensure that all 
stakeholders were included in the decision-making process for the Tech 
Prep marketing plan. 

Describe the activities your consortium planned to use to assure the 
participation of stakeholder groups in the planning and implementation 
of the marketing of Tech Prep programs. 

The need for including a wide variety of stakeholders in the planning and 
implementation phase of marketing Tech Prep courses and programs was 
recognized by all consortia. Consequently, all consortia reported the participation 
of a broad group of constituents Through joint conferences, workshops, planning 
meetings, focus groups, or committee memberships, consortia leadership and 
steering committees ensured that district and school-level staff, instructors, 
technical college partners, parents and community groups, and, perhaps most 
importantly, business and industry, were represented and participated in 
marketing. One consortium reported that it ensured that all consortium activities 
addressed the needs of special populations. 



Implementation (What was actually done?) 

Describe the involvement that stakeholder groups had in the marketing 

of Tech Prep programs in your area. 

Consortia attempted to market Tech Prep through active involvement of all 
stakeholder groups that had participated in the planning phase: they developed 
and distributed promotional materials, such as flyers and brochures that had 
been endorsed by business and industry and other stakeholder groups; they 
broadcast radio ads and ran newspaper ads that spoke to the broad involvement by 
all stakeholders; and they made joint presentations to parents, students, and the 
community at large. Several consortia reported that the broad involvement in 
Tech Prep resulted in site visits by business and industry leaders to the schools, 
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or, conversely, in ‘externships’ and visits by students and instructors to local 
business and industry sites. One consortium remarked on successfully involving 
the local Chamber of Commerce. Another pointed to the promotion of SCANS 
information by a Private Industry Council. 



Continuous Improvement 

How were you able to determine if the participation of stakeholder groups 
in the marketing of Tech Prep programs was successful? What data did 
you collect? 

How are you using this information to improve the future participation of 
stakeholders in marketing Tech Prep programs in your consortium? 



Feedback regarding the effectiveness of these efforts was sought mostly in the 
number of students participating in Tech Prep and the number of Tech Prep 
graduates. In some instances, consortia tracked the number of joint meetings. 
All consortia used informal or semi-formal feedback, such as participant 
assessments collected at the end of meetings, workshops, or seminars. This 
information was used to assess and improve the quality of ongoing marketing 
activities, and to improve the process of collaboration among different stakeholder 
groups. 



MRDC, University of Minnesota © 1996 



47 



Page 45 



Tech Prep System: Marketing 



Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Marketing 
System Activity - Evaluation 



Planning 



Describe how the strategies to evaluate the success of your consortium’s 
marketing plans for Tech Prep were developed. 

Describe the assessment strategies your consortium planned to use to 
determine the success of its Tech Prep marketing activities. 

Planning for the assessment of a consortium’s marketing effort was an area that, in 
general, had not received much attention. Several consortia reported that no 
assessment strategy had been included during the planning phase, others said that 
they relied on informal ways of verifying whether marketing was successful. Where 
it had been addressed, numeric outcome measures were cited: the goal to have each 
school in the consortium sign at least one articulation agreement, an increase in the 
level of interest in Tech Prep, increased enrollments, Tech Prep certificates awarded, 
or enrollment in post-secondary with advanced standing. Two consortia had defined 
a positive change in students’ and parents’ attitudes toward Tech Prep as indicators 
of successful marketing. Another consortium was concerned with a growth in 
stakeholder satisfaction with all program components as measured by feedback from 
steering committee members. 



Implementation (What was actually done?) 

Describe the assessment activities that were conducted by the consortium 

to determine the success of your Tech Prep marketing program. 

No consortium evaluated the success of its marketing effort in isolation from 
other measurements. Most consortia cited the use of enrollment data tracking as 
an indicator for successful marketing. Others tracked the volume and kind of 
promotional materials that schools and districts requested. There was also 
widespread use of the number of articulation agreements as indicators of success 
in marketing Tech Prep. One consortium had used a broad assessment strategy 
consisting of surveys, questionnaires, focus groups, enrollment data, and 
feedback from students, parents, and the business community. For several 
consortia, the self-assessment process served as focal point to reflect on their 
marketing efforts and assess the effectiveness of measures taken. 
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Continuous Improvement 

How were you able to determine if the assessment activities used to 
evaluate your marketing of Tech Prep programs were successful? What 
data did you collect? 

How are you using this information to improve the future assessment of 
your marketing program for your consortium? 

Many consortia reported that their assessment process for marketing was still in a 
developmental stage, since the program and associated marketing efforts were 
relatively new. Several cited looking for guidance from the state for the development 
of a process and instrument. There was widespread agreement that marketing was 
an essential component of Tech Prep, and that the goal was to increase awareness 
and commitment to Tech Prep, and to increase enrollments and certificates awarded. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Retrospective - Marketing 



Planning 



What planning processes worked best for your consortium while it was 
designing the marketing plan for Tech Prep programs? 



By far the most frequently mentioned factor in successful planning for marketing 
Tech Prep was the active involvement and committed work of a broad-based 
steering committee of representatives from the districts, technical colleges, the 
community, and business and industry. This involvement allowed for 
networking opportunities across boundaries, facilitated the exchange of 
information, and ensured that problems could be addressed in a timely and 
effective manner. Broad-based participation also facilitated and strengthened the 
drafting of articulation agreements. Further emphasis was placed on ongoing 
communication and comprehensive inservices for all staff involved in Tech Prep. 



Based on your experience, what pitfalls would you caution a new 
consortium about when planning to market Tech Prep programs? 



Constraints in terms of funds, time, and resources plagued most all consortia. Many, 
therefore, cautioned against taking on too many initiatives at once, overtaxing the 
system, or focusing too many resources on big promotional efforts, such as video or 
radio/TV ads. Consortia also advised developing a strategic plan and adhering to it, 
in order to promote a common vision of Tech Prep, to involve key individual from 
different levels, and to consider the involvement of all stakeholders from the very 
beginning. Several consortia cautioned against implementing a marketing strategy 
before both programs and processes are in place. 
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Implementation 

When implementing your Tech Prep marketing plan, in which areas of 
activity did you experience the greatest successes? Explain why you 
think these occurred. 



Among the successful activities surrounding the marketing of Tech Prep was the 
involvement of teachers, counselors, and other site-based staff. Several consortia 
identified word-of-mouth promotion of Tech Prep as by far the most effective form 
of marketing. To this extent, it was necessary to provide instructors and 
counselors with accurate and timely information. This aspect, too, was 
successful. Promotional printed materials, such as brochures, booklets, and 
flyers, also helped to promote Tech Prep. Another successful aspect was the 
quality of the program. Tech Prep courses that are seen as being on the cutting 
edge of technology, as one consortium put it, will sell themselves, and students 
will be anxious to enroll. A different aspect that was seen as a success was the 
networking opportunities provided by getting several stakeholders involved. This 
led to events such as teachers’ tours of local businesses and industry 
representatives speaking before students, providing realistic and accurate 
information and helping students select Tech Prep programs. 



Please describe the three of four greatest difficulties you have 
encountered in marketing your Tech Prep programs? Please explain 
why you think these occurred. 



Among the biggest barriers to successful marketing was the negative attitude toward 
Tech Prep and Vocational Education, and, conversely, the perceived superiority of an 
academic curriculum and a four-year college degree. Students and parents alike still 
lack adequate information about the value of Tech Prep. Another barrier that was 
addressed frequently was the lack of adequate funding, resources, and time, and the 
difficulty in coordinating the activities of several school districts, especially in large 
consortia. Other difficulties were a competition of Tech Prep with private business 
schools that are experienced in marketing to a broad audience, and the lack of 
involvement by middle schools in Tech Prep. 
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Continuous Improvement 



Which areas of activity during the implementation of your Tech Prep 
marketing program were most easily improved? Why? 



Many consortia found the addition of courses, the drafting of articulation 
agreements, and the expansion of the curriculum to include applied academics 
easy to accomplish. The improvement of the level of communication between 
secondary and post-secondary schools was also mentioned several times. 
Another aspect that consortia addressed as positive was that slowly the public 
perception of the value of Tech Prep was beginning to change. Tech Prep is 
gaining in popularity in many schools. The concept is beginning to take hold, 
leading to improved recognition of its value. This is due, in part, to staff 
inservices, frequent communications with parents and students, and a large 
number of public presentations. 



Please describe the areas you found most difficult to improve in the 
marketing of Tech Prep in your consortium and explain why you think 
this was so? 



Among the biggest challenges continues to be the misperception of the value of Tech 
Prep as an inferior alternative to an academic education. This change in attitude 
needs to take place in teachers and counselors as well as in students, parents, and 
the community. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Overall Planning 



Planning 



Describe the mechanisms you planned for implementing Tech Prep 
student assessment and assessment goals from your consortium’s 
original strategic plan (e.g., committees, planning groups)? 



Describe the overall goals, activity plans, and timelines with which you 
charged personnel assigned to develop Tech Prep student assessment 
and assessment during the project. 

Student assessment and program assessment is at the core of a Tech Prep 
program and thus is a key component of every consortium’s strategic plan. The 
planning of assessment and assessment strategies was typically conducted by a 
central sub-committee or team reporting to the consortium steering committee. 
The assessment committee’s tasks included the development of an assessment 
design, data collection instruments, and processes for data collection, analysis, 
and reporting. Evaluation typically included both outcome (summative) and 
process (formative) indicators. In several cases, assessment criteria were 
developed with the help of external criteria, such as the Minnesota Graduation 
Rule, SCANS competencies, CORD materials, and defined district learner 
outcomes. Evaluation criteria were coordinated with post-secondary partners, 
and included course-by-course performance-based authentic assessments, 
entrance or placement tests of partnering technical colleges that helped to guide 
curriculum content at the secondary level, and follow-up studies of high school 
graduates. 
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Implementation 

Describe how the goals, activity plans, and timelines for student 

assessment and assessment Tech Prep were coordinated among the 
institutions within your consortium? 



While many consortia had extensive plans for assessment and assessment, the 
implementation of these plans is, in many cases, still unfolding. In all consortia, 
the assessment teams or committees have initiated meetings and discussions 
around assessment tools and approaches with various stakeholders. To that end, 
information has been disseminated, workshops and other events have been held, 
and site visits by consortium leadership have been conducted. One consortium is 
in the process of piloting an assessment plan with a number of pilot schools, 
another reported that all member schools have initial assessment plans for 
students and courses in place. In one instance, the district school collaborated 
with the technical college on the testing and reporting of student competencies in 
Tech Prep related courses. Another reported the use of the MRDC process as the 
primary tool for assessment and assessment. A number of quantitative measures 
are also in place in some consortia. These include the tracking of certificates 
awarded to Tech Prep students, the number of articulated courses, enrollment 
data, and tracking of transferring high school graduates. Informal information 
are also collected in many consortia, including feedback from teachers, students, 
counselors, and parents. 



Continuous Improvement 

How were you able to determine if the mechanisms, goals, activity plans 
and timelines set for Tech Prep student assessment and assessment 
were reasonable; what data did you collect? 

How are you using this information to develop new or revised overall 
goals and strategies for implementing the consortium's Tech Prep 
student assessment and assessment? 

The overarching goal of assessment and assessment is the improvement of Tech 
Prep programs and services. To this end, some consortia conduct annual reviews 
of articulated course assessments, and provide summary information to 
consortium leadership regularly. The importance of assessment and assessment 
appears to be well understood, but there is a lack of time and resources for 
comprehensive assessment in many consortia. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Staff Development 



Planning 



Describe the process your consortium used to plan for the staff development 
required to conduct Tech Prep student assessment and program 
assessment. 

Describe the staff development activities your consortium selected to prepare 
staff to conduct Tech Prep student assessment and program assessment. 

Planning for staff development around Tech Prep student assessment was either 
centralized by consortium leadership or conducted locally by individual districts. 
When done locally, close collaboration between the district and the consortium 
ensured some degree of oversight. The planning typically addressed two aspects: 
selection of student assessment instruments and program assessment methods, and 
staff training in those tools and methods. Virtually all consortia involved a range of 
stakeholders in the planning process, including counselors, technical college staff, 
students, and parents. Participation of diverse groups took the form of workshops, 
site visits by coordinators, and meetings. The student assessment tools selected 
included authentic assessment, the Minnesota Graduation Rule, Guidance 
Counseling, learning style inventories, and standard assessments in math, science, 
and communication. Once these methods and tools had been decided upon, consortia 
conducted needs assessments and gap analyses to determine the development and 
training needs of staff and counselors. One consortium had a specialized assessment 
available for this task, another conducted a random sample survey to assess the 
extant skill level of its staff. 



Implementation (What was actually done?) 

Describe the activities your consortium conducted to train staff in Tech Prep 
student assessment and program assessment. 



While two consortia reported that no formal training in Tech Prep student 
assessment and program assessment had been conducted, the other eight consortia 
cited a variety of staff development activities, ranging from in-service training to 
workshops, teleconferences, and sponsorship of participation in state and national 
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events, such as the National Tech Prep Conference, the AVA Annual Conference, 
The National Counselors’ Teleconference, or events organized by the Portland Area 
Vocational Education Consortium (PAVTEC). Several consortia held cross-training 
events with technical college and state university instructors, or used joint 
development (“Train by Sharing”) methods. Staff training focused on the use of 
assessment tools and methods, and included the use of the Tech Prep Identifier 
Form, the Minnesota High School Follow-up System, authentic assessment, 
articulation, the Minnesota Graduation Rule, and career planning. 



Continuous Improvement 

How were you able to determine if the staff development activities you 
conducted were appropriate to Tech Prep student assessment and 
program assessment needs? What data did you collect? 

How are you using this information to improve future staff development 
activities for student assessment and program assessment in your Tech 
Prep consortium? 

Virtually all consortia reported the use of informal feedback as the primary tool to 
determine the success of staff development activities. This included assessments of 
training events by participants, but also perceived changes in the level of interest by 
students and parents in Tech Prep. Typically, this information was shared with 
consortium leadership. Several consortia tracked the number of Tech Prep students 
and viewed an increase in Tech Prep enrollment as evidence of successful staff 
development. Most consortia expressed the need for a more comprehensive and valid 
assessment and program assessment. While several consortia look to the state for 
such a system, others reported the use of existing data collection activities, such as 
the Minnesota High School Follow-up system, the High School Senior Survey, and the 
Tech Prep Consortia Evaluation System. There is an interest in the development and 
use of career portfolios, and a need for the creation of increased career awareness by 
students beginning as early as elementary school. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Special Populations 



Planning 

Describe the process your consortium used to plan Tech Prep student 
assessment and program assessment strategies for students from 
special populations. 

Describe the strategies your consortium selected for Tech Prep student 
assessment and program assessment for special populations. 

The planning for Tech Prep assessment of students from special populations 
involved, in almost all cases, professional staff familiar with this population, namely 
transition coordinators, LD, EBD, and ESL departments, and special needs students 
counselors and case workers. The purpose of their involvement was to ascertain that 
the assessment and program assessment tools and processes accommodated the 
needs of special populations. The outcome of this deliberation was in all cases the 
decision to use either processes and tools already in existence for special populations, 
or to use the same methods for all Tech Prep students, irrespective of status. In 
either case, consortia made an effort to accommodate the needs of special populations 
by making special arrangements in terms of scheduling of and access to assessment 
activities. 



Implementation (What was actually done?) 

Describe how the performance of Tech Prep students from special 
populations was assessed in your consortium. 

Describe how the success of Tech Prep programs in sexwing students from 
special populations was evaluated. 



Consortia described the process and methods used to assess the performance of Tech 
Prep students from special populations. In one case individual schools contracted 
with outside agencies for aptitude testing and involvement of special education staff 
and programs. In another instance, the consortium reported a team approach 
involving special education staff. Many consortia used PER and IEP processes to set 
goals and track performance of special populations. This involved, in several 
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consortia, special population counselors or case workers. One consortium reported 
work with incarcerated students that had been funded through a Tech Prep grant. 
Overall, many consortia reported the use of informal methods, such as verbal 
feedback and testimonials, to gain insight into the success of special populations in 
Tech Prep programs. 



Continuous Improvement 

How were you able to determine if the Tech Prep student assessment and 
program assessment strategies were successful? What data did you 
collect? 

How are you using the information you collected to develop new or revised 
student assessment and program assessment strategies? 



The type of data collected mirrors those collected for regular students. Consortia 
tracked the number of enrollments in Tech Prep, the number of certificates awarded, 
the number of transfers to technical colleges, and satisfaction ratings of students, 
parents, and teachers. This information was collected, in many cases, through 
special populations staff, such as case managers or counselors. The information was 
used to track performance and detect early indicators of possible failure in the 
program, to decide upon the need of supplemental materials, and to provide career 
assessment and counseling services. Overall, there was consensus that the 
assessment of special populations was in its initial stage. Early results, however, 
pointed to a decrease in the trend of disruptive student behavior, but also pointed to a 
need to Create more awareness of Tech Prep among special education teachers, to 
develop an awareness among minority students of work place requirements, and to 
encourage enrollment in non-traditional areas. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Curriculum Integration 



Planning 

Describe how your consortium developed a plan to conduct student 

assessment and program assessment within an integrated curriculum. 

Describe the unique characteristics of your student assessment and 
program assessment plans that resulted from the consideration of 
curriculum integration in Tech Prep programs. 



In virtually all consortia, planning for student assessment and program assessment 
within integrated curricula was conducted centrally by consortium leadership or a 
committee appointed by the consortium. In most cases, however, a variety of 
stakeholders, such as business, industry, labor, member districts, and post- 
secondary institutions, were consulted or played key roles on the committee. Only one 
consortium left the planning entirely to the individual member districts. The 
planning committee’s tasks were to develop assessment strategies and decide upon 
assessment methods, tools, and instruments. In one instance, the consortium 
reported that the committee’s initial activity consisted of developing a global definition 
of integration for its member districts and schools. Many consortia used outside 
resources, such as CORD materials, or the ASSET entrance/placement test as models 
to guide their work, others sought information from out-of-state Tech Prep consortia 
or the National Center for Research in Vot ational Education (NCRVE). There was 
wide-scale agreement over the goals of integration assessment and assessment, 
namely to provide the consortium and districts continuous information about the 
status of curriculum integration and to be able to modify and adjust the programs in 
a timely fashion. 



Implementation (What was actually done?) 

What unique student assessment and program assessment techniques were 
adopted to accommodate an integrated curriculum in your consortium? 



The consortia varied in the level of actual implementation of student assessment and 
program assessment techniques. Some had in fact developed and implemented 
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specific instruments and techniques (for example tests, students projects, 
demonstration of skills, pre/post designs in applied academic areas). Others relied 
primarily on standardized tests, such as the ASSET entrance/placement test. A few 
were only now beginning to implement their plans. Related outcomes of those who 
had implemented instruments and techniques were a variety of materials and 
information provided to students, parents, and the community. One consortium, for 
instance, had developed and distributed a brochure about Tech Prep classes and 
careers. Others had begun to regularly advise students about the progress of 
articulation agreements. Yet others had school counselors heavily involved in 
identifying students for Tech Prep and promoting the program. 



Continuous Improvement 

How were you able to determine if the student assessment and program 
assessment activities you developed in light of curriculum integration 
were appropriate? What data did you collect? 

How are you using this information to improve your student assessment 
and program assessment systems in relation to curriculum integration? 



The type of data collected for determining the success of student assessment and 
program assessment activities in light of curriculum integration consisted primarily 
of enrollment and persistence data. Consortia tracked the number of students in Tech 
Prep courses, used existing instruments like the Minnesota High School Follow-up 
system, the Tech Prep Identifier Form, and locally used standardized tests. A 
number of consortia also collected informal feedback from students and instructors — 
both at the secondary and post-secondary levels, and measured the level of interest in 
applied and integrated courses. The information was regulatory fed back to the 
consortium level where executive committees used the information to decide over 
curriculum acquisition matters. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Articulation 



Planning 



Describe how your consortium developed a plan to conduct student 

assessment and program assessment within an articulated curriculum 
between secondary and post-secondary education. 

Describe the unique characteristics of your student assessment and 
program assessment plans that resulted from the consideration of 
articulated programs. 



The mandate to conduct students assessment and program assessment within the 
articulated curriculum between secondary and post-secondary levels was clearly 
understood by all consortia, and planning efforts were in process in most. Many 
consortia had established subcommittees consisting of members of the secondary and 
post-secondary communities whose task was to decide upon criteria for successful 
articulation of programs and courses, and to develop a process for collecting and 
analyzing assessment and assessment data. Some consortia used benchmarking as 
a way of learning about successful assessment procedures in other consortia. There 
appeared to be consensus among the cohort that the assessment should span the 
entire four year course of study. Actual planning, however, differed from consortium 
to consortium. Some had not progressed much beyond initial meetings and 
discussion, while others had succeeded in defining and implementing specific 
course-by-course performance-based authentic assessments for most of the 
articulated courses. Many plan to review their assessment and assessment 
approaches annually, again involving members of secondary and post-secondary 
levels. Consortia also recognized that any assessment strategy should be 
communicated to counselors, teachers, and students, and several had plans to 
disseminate such information. 
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Implementation (What was actually done?) 

What unique student assessment and program assessment techniques were 
adopted to accommodate an articulated curriculum in your consortium? 



Implementation of assessment and assessment was, in most cases, still in its initial 
stage due to the fact that only a few Tech Prep students have graduated from high 
school and had the opportunity to enroll at the post-secondary level. For many 
consortia, then, it was too early to have implemented student assessment and 
program assessment to any large extent. Progress, however, had been made in 
meeting with representatives of both levels to determine which courses and programs 
were to be articulated, what objectives and requirements those courses had, and how 
those courses could best be streamlined. In several instances, this effort led to the 
development of written guidelines that specifically recommended sequences of 
courses and provided detailed entrance requirements for programs/courses at the 
post- secondary level to secondary students, teachers, counselors, and parents. 
Similarly, inservices and staff development activities had been carried out to provide 
current and detailed information to staff about the progress and implementation of 
articulation efforts. One consortium was working on a process to allow students 
entering from other consortia's secondary systems into its own post-secondary 
system. 



Continuous Improvement 

How were you able to determine if the student assessment and program 
assessment activities you implemented in light of an articulated 
curriculum were appropriate? What data did you collect? 

How are you using this information to improve your student assessment 
and program assessment systems in relation to curriculum 
articulation? 



Due to the recent implementation of Tech Prep, none of the consortia had a solid 
process for evaluating the effectiveness of its in place assessment system. Most 
reported tracking the number of articulated courses and programs, the number of 
enrollments, the number of Tech Prep secondary certificates awarded, and the 
number of students who continue on at the technical college or community college 
level. Many consortia had decided to review their articulation agreements annually, 
to review the approval of articulated courses annually, and to decide upon revisions or 
changes jointly with secondary and post-secondary members. Several consortia 
mentioned the plan to incorporate the requirements of the Minnesota Graduation 
Rule standards into the articulation assessments. Most consortia collect informal 
feedback and report this information back to the steering committee. One consortium 
observed the need to move away from a view of articulation as a time-shortened 
approach to articulation as a higher skills concept. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Partnerships 



Planning 



Describe the process your consortium planned to ensure that all 
stakeholders were included in the design of Tech Prep student 
assessment and program assessment. 

Describe the activities your consortium planned to use to ensure the 

participation of stakeholder groups in the planning and implementation 
of the student assessment and program assessment systems in your 
Tech Prep programs. 



Forming partnerships and collaboration among and between levels and sectors is a 
key aspect of Tech Prep. All consortia reported, consequently, the involvement of a 
broad spectr um of constituents in the planning of Tech Prep overall, and in student 
assessment and program assessment in particular. Through the involvement of 
students, staff, schools, the community, business and industry, and outside agencies, 
consortia developed plans to conduct assessment and assessment. In one case, a 
grant made the involvement of instructional staff of a correctional facility possible. 
Involvement of constituents also mandated communication strategies that kept 
everybody informed regularly and provided a means for input and involvement. In 
one consortium this was part of the responsibilities of newly appointed high school 
site coordinators. Another consortium reported the plan to review all articulated 
curricula course-by-course at technical college advisory committee meetings. 



Implementation (What was actually done?) 

Describe how members of stakeholder groups were actually included in the 
design and implementation of the assessment of Tech Prep students and 
Tech Prep program assessment. 



Consortia solicited involvement of a wide variety of stakeholders primarily through 
public events, such as dinners, ‘Tech Prep Suppers’, workshops, seminars, and 
regular meetings. One consortium attempted to recruit members for focus groups 



O 

ERIC 



MRDC, University of Minnesota © 1996 



63 



Page 61 



Tech Prep System: Student Assessment and Evaluation 



via newspaper ads. Other ways of ensuring the involvement of stakeholders were 
staff development events with participants from a variety of organizations and 
institutions, invitations to workshops on assessment and assessment held by a 
nationally known Tech Prep expert, and wide membership in steering committees, 
articulation groups, and advisory committees. These activities have resulted in the 
formation of long-lasting partnerships among members who regularly address 
assessment and assessment issues. The most clearly articulated result came from 
one consortium that had established guidelines and standards for assessment and 
assessment, had validated these guidelines through members from business and 
industry, and bad technical college advisory committees review and approve the 
assessments. As in the planning stage, continuous and timely communication to all 
stakeholders was seen as critical for effective partnering. 



Continuous Improvement 

How did you determine if the inclusion of stakeholders in the design and 
implementation of student assessment and program assessment was 
appropriate and successful? What data did you collect? 

How are you using this information to improve the contribution of members 
of stakeholder groups in the assessment of Tech Prep students and the 
assessment of Tech Prep programs? 

All consortia reported collecting informal feedback about the various activities 
regularly and modifying processes and techniques accordingly. In addition, several 
consortia collect feedback from those affected by Tech Prep: students, parents, 
business and industry, and the community at large. Business involvement in the 
assessment process is seen as critical, as is the need to capture the voice of students 
currently enrolled in Tech Prep. One consortium relied heavily on the self- 
assessment process and the information collected through it. Another consortium 
reported that the articulation process and partnering with post-secondary was seen 
as successful, and that it supported the use of authentic assessment as a 
measurement tool. Several consortia stressed the importance of continued business 
involvement in assessing the relevance of the Tech Prep curriculum and the skills 
exhibited by Tech Prep graduates once they are employed. Other suggestions were the 
establishment of parent and business advisory boards, and the capturing of student 
opinions. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Student Assessment & Evaluation 
System Activity - Evaluation 



Planning 



Describe the process your consortium used to plan for the assessment of the 
Tech Prep student assessment and program assessment strategies. 

Describe the assessment strategies your consortium selected to appraise its 
Tech Prep student assessment and program assessment systems. 



The planning for an ‘assessment of the assessment process’ was conducted, in most 
cases, by consortium leadership, although one consortium reported leaving it up to 
the individual districts, and several others indicating that they are looking to the state 
for guidance. The development of criteria for this purpose was typically the task of 
articulation committees or other central committees or groups. In most cases, the 
development of specific assessment strategies was still in its formative stages. 



Implementation (What was actually done?) 

Describe the activities your consortium actually conducted to appraise its 
Tech Prep student assessment and program assessment activities. 



Most consortia relied on informal information, such as feedback from instructors, 
counselors, and students, to appraise the assessment and program assessment 
activities. 
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Continuous Improvement 

How were you able to determine if the procedures used to evaluate your Tech 
Prep student assessment and program assessment activities were 
successful? What data did you collect? 

How are you using this information to improve the future assessment of the 
student assessment and program assessment activities for your 
consortium? 



Informal information is used at annual review meetings of articulation agreements 
and fo/ other planning purposes. In one instance, a consortium reported that a new 
assessment process was under development because feedback from a variety of 
stakeholders indicated that the current practice of collecting outcome-based data was 
not supported by the member schools and districts. Another consortium indicated the 
need for consortium leadership to standardize the student assessment and program 
assessment process. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Retrospective - Student Assessment & Evaluation 



Planning 

WJiat planning processes worked best for your consortium in designing 
Tech Prep student assessment and program assessment systems? 

The consortia stressed two main areas that had proven to be successful in planning 
for student assessment and program assessment. Most reported that a team 
approach with involvement of many different stakeholders created a level of synergy 
and commitment that helped the process along. A related comment was the creating 
of good rapport, trust, and direct and open communication among all parties involved 
in the planning process. Only one consortium reported that, due to its large size, a 
decentralized approach, letting local districts decide upon their own process and 
tools, worked best. The second factor that was frequently addressed was the 
importance of having a leadership team that was experienced or had access to expert 
resources in the area of assessment and assessment. One consortium reported that it 
had successfully involved high school coordinators who had expertise and knowledge 
of local circumstances in the planning process. 



Based on your experience, what pitfalls would you caution a new 

consortium about when planning Tech Prep student assessment and 
program assessment? 



In addressing potential pitfalls to avoid in planning for assessment and program 
assessment, several consortia strongly advised to involve from the very beginning 
‘front-line’ staff, i.e. teachers, counselors, and other who have detailed knowledge of 
the specific district and school. To this end, the mandate to assess students and 
program must be communicated early in the planning process. A second concern 
was related to the integration of concurrent policy changes, rules and movements 
other the Tech Prep, such as the high school graduation rule, into the Tech Prep 
assessment process Thirdly, several consortia warned not to underestimate the 
amount of time and resources required for successful planning and implementation 
of Tech Prep assessment. The merger of the technical college system with higher 
education was reported to create additional problems for Tech Prep. These resource- 
related problems were acutely felt by large consortia that incorporate a large number 
of diverse and geographically distant districts. A last point of warning was related to 
the difficulty finding or developing valid instruments and methods of assessment and 
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assessment, especially when a triangulated approach, using several different 
methods of assessment and assessment, is desired. 



Implementation 

When implementing Tech Prep student assessment and assessment, in 
what areas of activity did you experience the greatest success? Why? 

The most frequently cited success in implementing student assessment and program 
assessment for Tech Prep was that the process of involving a variety of stakeholders 
led to good working relationships among the different groups and stakeholders. This 
resulted in improved communication, effective decision making processes, a general 
atmosphere of trust, and, ultimately, solid assessment and assessment procedures 
that were supported by all. Another benefit was the number and quality of 
articulation agreements which attracted a lot of attention from students, parents, and 
the community at large. The process resulted in a shift in thinking about 
instructional practice not only among teachers, but also students, parents, 
counselors, and career center staff who all acted in support of Tech Prep. 



What Tech Prep student assessment and program assessment activities 
would revise if you had it to do over again? 



Among the activities that consortia would revise were the active involvement of 
business and industry early in the assessment planning process, and the greater 
education of parents and students about changes in the economy, the work force, the 
labor market, and the educational requirements for success in today’s world. 
Consortia also addressed the need to create assessment and assessment tools and 
processes early in the program and apply it uniformly in all districts; or, at least, to 
standardize existing assessment procedures within the consortium. A final point of 
suggestion was to provide training that would help teachers tie performance based 
assessment to graduation rule standards. 



Continuous Improvement 

Which areas of activity during the design and implementation of Tech Prep 
student assessment and program assessment systems were the easiest 
to improve? Please explain why you think they were? 

The areas that were easiest to improve were the cooperation between institutions 
because secondary and post- secondary schools share the common vision of providing 
a quality education. Articulation agreements, in particular, were easy to improve 
with the support of the technical college. Areas were easy to improve to the extent 
that little or no financial investment was required and when staff had a large degree 
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of control over that particular area. The formation of smooth and rewarding working 
relationships between different levels of institutions was further cited as contributing 
to the review process for articulated assessments. Lastly, several key measures, such 
as the number of completed articulation agreements and the Tech Prep Course 
Survey instrument used by one consortium, were easy to obtain and useful for 
program assessment and assessment purposes. 



Describe the areas you found most difficult to improve in Tech Prep student 
assessment and program assessment. Please explain why you think 
they were. 

A major area of difficulty for consortia was to overcome the stereotypical judgment of 
Tech Prep as inferior to an academic education. Even internally, support for Tech 
Prep was oftentimes difficult to obtain, either because information had not been 
shared from the start, or because stakeholders are involved in a myriad of other 
activities and simply have little time for Tech Prep. Many consortia also reported 
difficulties with gathering and interpreting assessment information. Evaluation data 
were seen as difficult to gather early on in the implementation of the program, or they 
were hard to gather in a comprehensive fashion as baseline data against which to 
compare later achievement. In many cases, consortia lacked the resources or 
expertise for developing valid and comprehensive assessment tools. One consortium 
also reported that assessment data were oftentimes seen as too removed from more 
pressing activities on hand. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Overall Planning 



Planning 



Describe the mechanisms you planned for implementing Tech Prep 
support services and counseling service goals from your 
consortium’s original strategic plan (e.g., committees, planning 
groups)? 

Describe the overall goals, activity plans, and timelines with which you 
charged personnel assigned to develop Tech Prep support services 
and counseling services during the project. 



Support services and student counseling were activities that were included in 
most consortia’s strategic plans. Planning for those activities, therefore, received 
priority in many cases. The overall goals for this area included the provision of 
support and services that enable students to make informed decisions about 
career choices, to provide for career awareness and exploration, and to put 
processes in place that would lead to systematic planning and monitoring of 
students’ educational activities. To this end, many consortia had planned the 
development and dissemination of information materials, such as brochures, 
flyers, or career option information. Planning for the development of these 
materials and activities was done jointly between secondary and post-secondary in 
many cases. Most consortia also stressed the importance of involving school and 
college counseling staff in the planning activities. Staff development played a big 
role in this area, as teachers, counselors, administrators, and other staff needed 
to be well informed of Tech Prep options and activities. Several consortia 
expanded their planning horizon to include career development activities in 
junior high school. Many emphasized the need for inclusion and accommodation 
of special needs students. Another key point with several consortia was the 
linkage between counseling and industry and business needs. 
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Implementation 

Describe how the goals, activity plans, and timelines for the Tech Prep 
support services and counseling services were coordinated among 
the institutions within your consortium? 

Most consortia emphasized that support services and student counseling were 
ongoing activities that required continuous or regularly scheduled inservices and 
updating so that staff and personnel could stay current as Tech Prep 
implementation progressed. Initial preparation for staff and personnel included 
in-depth workshops and inservices, participation in seminars and conferences, 
and one-on-one coaching. In many cases, there was a consortium-wide 
implementation team with participation of counselors from member districts and 
schools. This team set guidelines and parameters, but typically left the 
implementation and modification of generic models to the individual districts and 
sites. In most cases printed materials, such as brochures or career portfolios, 
had been developed, distributed, and were in use. Several consortia had produced 
or purchased non-print materials, such as a video series on career exploration or 
the hosting of a teleconference on career counseling. By and large, the primary 
emphasis in most consortia was to adequately prepare counselors and staff to 
carry out their roles within a Tech Prep program. 



Continuous Improvement 

How were you able to determine if the mechanisms, goals, activity plans 
and timelines set for Tech Prep support services and counseling 
services were reasonable; what data did you collect? 

How are you using this information to develop new or revised overall 
goals and strategies for implementing the consortium's Tech Prep 
support services and counseling services? 

Many consortia used enrollment information as primary evidence of the 
effectiveness of student support and counseling. Consortia, therefore, tracked 
student enrollment, the number of Tech Prep diplomas awarded, and student 
performance in Tech Prep courses. Other measures included the level of interest 
and the number of requests for information materials. Much emphasis was also 
placed on collecting and analyzing informal information and feedback from 
counselors at seminars, workshops, or other inservice activities. Several themes 
emerged from this informal feedback: greater emphasis needs to be placed on 
including counselors in planning for support and counseling; counseling and 
awareness building needs to begin as early as junior high school; and there is a 
need to include the business community in order to stay current on business and 
industry demands and conditions. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Staff Development 



Planning 

Describe the process your consortium used to plan staff development activities 
in the area of support services and counseling for Tech Prep students. 



Describe what activities were selected to prepare staff to conduct Tech Prep- 
related support services and counseling. 

The planning activities for staff development activities in the area of student support 
and counseling were conducted at the consortium level with regular participation of 
counselors and other staff. In several instances, the team conducted a needs analysis 
to determine development needs of staff at the local level. In other consortia, the team 
sought information about counseling methods and Tech Prep from workshops on 
career counseling and job trends of the future. Some teams visited local businesses in 
order to gain an accurate assessment of employer needs and to be able to modify 
existing counseling and support activities. Participation in the planning phase was 
ensured through counselor representation on teams and, in one case, through the 
consortium coordinator’s visits to counselors at local high schools. 



Implementation (What was actually done?) 

Please describe the staff development activities your consortium actually 
conducted to insure appropriate Tech Prep support services and 
counseling. 

Staff development was conducted primarily through workshops, seminars, 
participation (in person or televised) at local, state, and national conferences, and 
other inservice events. Several consortia had invited nationally known experts on 
career counseling. Typically, a cross section of Tech Prep stakeholders, including 
teachers, community members, and business and industry representatives were 
invited to these events. Other activities included the distribution of regular 
newsletters and other informational materials that would allow counselors to self- 
study and keep up with the implementation of Tech Prep in their districts. Yet 
another way of educating staff and counselors on Tech Prep was to participate in 
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planning meetings and to visit other districts and consortia in order to glean best 
practices and exchange information. 



Continuous Improvement 

How were you able to determine if the Tech Prep support services and 

counseling staff development activities were successful? What data did 
you collect? 

How are you using this information to develop or revise staff development 

activities for implementing Tech Prep support services and counseling? 



The data collected to assess the results of staff development efforts were primarily 
informal. Evaluations following inservice events and feedback from counselors and 
staff were used to modify the programs. Frequently, consortia reported on an 
increased level of interest by counselors and instructors, who, due to the high 
counselor/student ratio, oftentimes take on the roles of counselors. In addition, 
student enrollment data and the amount of informational and promotional materials 
requested were cited most often as evidence of successful staff development. A 
number of themes emerged from the information: career development and planning 
should start as early as possible, there is a need for better communication between 
counselors and administrators, several consortia expressed an interest in career 
portfolios, and many saw a need for increased flow and improved quality ol job 
forecasting information. { 
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Minnesota T .ch Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Special Populations 



Planning 

Describe the process which your consortium used to plan for support and 
counseling services specifically for special populations in Tech Prep. 



Describe the strategies your consortium selected to address the support services 
and counseling needs of Tech Prep students from special populations. 

There was unanimous agreement among the consortia that Tech Prep was a 
program for all students. Special populations should not be singled out in any way, 
but rather integrated with regular students. To this end, most consortia mentioned 
that the support and counseling services available to special populations for non-Tech 
Prep classes automatically carried over into Tech Prep. By and large, there was no 
mention of any special accommodation or activities designed for Tech Prep special 
needs students. Several consortia had special needs staff or advocates involved in 
overall planning. About one half of the ten consortia, however, addressed issues of 
gender equity in this context and reported attempts to attract more women into non- 
tradition al careers. 



Implementation (What was actually done?) 

Describe the activities you conducted to ensure that the support and counseling 
services addressed the needs of special populations in your Tech Prep 
programs? 



As with non-Tech Prep courses, the primary vehicle for providing support and 
counseling to special populations was the IEP process. This included the recording of 
Tech Prep classes taken, regular meetings to monitor progress and assist when 
needed, and the possibility to assess successful completion of Tech Prep courses by 
members of special populations. Beyond this, one consortium reported holding a 
workshop for school district counselors to discuss the needs of special populations, 
and another consortium organized a variety of activities designed to attract more 
women to technical Tech Prep careers. These activities included workshops, special 
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emphasis during campus tours for prospective students, a summer camp, and a 
student fair introducing non-traditional career opportunities. 



Continuous Improvement 

How were you able to determine if the selected strategies successfully 
addressed the support service and counseling needs o Tech Prep 
students from special populations? What data did you collect? 

How are you using this information to improve the delivery of support services 
and counseling to Tech Prep students from special populations? 

Because of the limited scope of activities designed especially for special populations, 
very little assessment or data collection was reported. One consortium reported 
keeping track of the number of special needs students who successfully complete 
Tech Prep classes, another mentioned the IEP process as a convenient method of 
assessing the success of special needs students in Tech Prep classes. Many 
consortia, however, raised the issue of becoming more proactive in regards to 
including special populations in Tech Prep, while several others reported a concern 
with gender equity issues around Tech Prep. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Curriculum Integration 



Planning 

Describe how your consortium developed a plan to provide support services and 
counseling within an integrated curriculum. 



Describe the unique characteristics of the support services and counseling 
services selected that resulted from the consideration of curriculum 
integration in Tech Prep programs. 



The planning process for the provision of support services and counseling regarding 
the integrated curriculum went hand-in-glove with the planning for the integration 
process. With the overall aim of promoting the integrated curriculum, most 
consortia placed a strong emphasis on including counselors from local school sites in 
the planning process for integration. This involvement of counselors resulted in 
links with local business and industry, such as industry representatives speaking to 
groups of counselors, counselors touring local business, and, in general, providing 
avenues for input from local industries in the planning process for integration. A 
second theme in the planning process was a strong emphasis on systematic career 
planning efforts in combination with raising awareness of the newly integrated 
curricula. Third, counselors often were seen as providers of the critical linkage 
between consortium-based planning efforts and local staff, students, and instructors. 
In this role, counselors provided input from the member schools to the consortium, 
but also conveyed information about integration planning from the consortium to 
their schools. 
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Implementation (What was actually done?) 

What unique student support and counseling services were adopted to 
accommodate an integrated curriculum in your consortium? 



Many consortia developed and distributed written information about the integrated 
curriculum, such as brochures, handbooks, and student guides that contained 
details about Tech Prep, provided graphics that illustrated the results of the 
integration process (matrices of articulated classes), and provided guidance for 
course planning. In one instance, the consortium had developed a guide with a 
planned and recommended sequence of courses for the four year Tech Prep 
curriculum and included both articulated and applied courses at both levels, 
secondary and post-secondary. Other consortia used students career portfolios, 
career fairs, advisor/advisee programs, and career centers and assessment software 
to counsel and guide Tech Prep students. In general, the role of counselors was 
described as critical to spreading the word about Tech Prep to students, parents, and 
instructors, identifying potential Tech Prep students, and providing detailed 
information about the programs and course/career options. 



Continuous Improvement 

How were you able to determine if the support and counseling services you 

implemented in light of curriculum integration were appropriate? What 
data did you collect? 



How are you using this information to improve Tech Prep-related support 
services and counseling in the schools in your consortium? 

Several consortia indicated that curriculum integration was still in its initial stages 
and, therefore, little data had been collected to date. Other consortia emphasized the 
role of enrollment statistics as indicators of successful support and counseling 
activities. Many had collected informal feedback from counselors, students, parents, 
and instructors, and used this information to modify the process as appropriate. A 
number of consortia had identified specific areas that needed improvement. These 
included the need for additional applied academic courses, improved communication 
flow between consortium leadership and local schools, better career planning 
approaches and instruments, and the need for additional resources. One consortium 
stressed the importance of continuing successful practices, such as monthly 
counselor meetings, and industry-based field trips. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Articulation 



Plaining 

Describe how your consortium developed a plan to deliver support and 
counseling services within an articulated curriculum between 
secondary and post-secondary education. 



Describe the unique characteristics of your support and counseling service 
plans that resulted from the consideration of articulated programs. 



In most consortia, high school and technical college counseling staff were involved in 
the planning and development of articulation agreements at the consortium level. 
This helped to ensure that any initial skepticism and concerns on the side of the 
counselors could be addressed and overcome, and that counseling staff understood 
the details of the articulation process and agreements well. One consortium reported 
having spent a long time to develop a common vision for Tech Prep articulation with 
technical college and high school counselors. Another consortium began the 
planning process by inventorying all high school level courses offered in the 
consortium in cooperation with the counselors, and identifying those that could be 
articulated. 



Implementation (What was actually done?) 

What unique support and counseling service techniques were adopted to 
accommodate an articulated curriculum in your consortium? 



The consortia used a variety of ways to accommodate articulated curricula and 
ensure that information about articulation is available to students, parents, 
instructors, and counselors. All consortia developed and distributed written 
information about articulation. This included registration handbooks, curriculum 
guides, Technical Career Guides, and information sheets with descriptions of career 
paths. Inservices for counselors were also used frequently. One consortium 
distributed copies of all signed articulation agreements to counseling staff, another 
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had developed career and academic planning materials for students and parents. 
Several consortia reported organizing special events, such as career days and field 
trips by secondary students to the technical college. Many consortia stressed the 
importance of keeping counselors and students well informed of any new articulation 
agreements or changes within already existing agreements. 



Continuous Improvement 

How were you able to determine if support arid counseling services you 

implemented in light of an articulated curriculum were appropriate? 
What data did you collect? 

How are you using this information to improve your support and counseling 
systems in relation to curriculum articulation? 

Few consortia reported specific activities designed to determine the effectiveness of 
counseling services in implemented articulated curricula. Several indicated the 
collection of informal feedback and concerns. Others reported that they evaluated the 
effects of counseling in light of enrollment trends in Tech Prep. There were several 
areas of concern that consortia tried to address. They included improving the flow of 
information and communication to counselors to enable them to provide better 
services to students and parents; increasing the understanding of students and 
parents regarding articulation through improved communication; and the need to 
work more closely with community colleges within their consortium. One 
consortium reported initiating processes to review course work completed in 
neighboring consortia for transfer to its own system. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Partnerships 



Planning 

Describe the process your consortium used to develop a plan for ensuring that 
all stakeholders were included in the design of Tech Prep support and 
counseling services. 



Describe the activities your consortium selected to guarantee the participation 
of stakeholder groups in the planning and implementation of support 
and counseling services in your Tech Prep programs. 



All consortia emphasized the importance of involving a broad range of stakeholders 
in the planning of Tech Prep in general, and for student support and counseling in 
particular. Consortia sought to actively involve all member schools, counselors, 
instructors, staff, students, parents, community members, other community- based 
institutions and, most importantly, the business and industry communities. To this 
end, invitations to meetings, Tech Prep dinners, open houses, and other events were 
distributed. In several cases, the consortium members and school staff conducted on- 
site visits and field trips to local business and industry sites and invited business 
representatives to their sites. Other consortia invited business and industry leaders to 
present to their staff. The purpose was to gather input from all stakeholders involved, 
strengthen their commitment and buy-in, and improve the quality of the planning 
process. Many consortia also stressed their effort to disseminate information about 
the progress of planning activities to a wide audience in order to encourage further 
participation and keep communication channels open. 
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Implementation (What was actually done?) 

Describe how members of stakeholder groups were actually included in the 
design and implementation of support and counseling services. 

Stakeholder groups participated in the design and implementation of support and 
counseling services at regularly scheduled planning and review meetings at the 
consortium and district or school level. They played an active part in events such as 
career fairs, career awareness activities, and open houses. They were involved in the 
design and development of written information such as brochures and guidelines. 
They further took part in many inservice events, either as participants or as 
presenters. 



Continuous Improvement 

How did you determine if the inclusion of stakeholders in the design and 

implementation of support and counseling services was appropriate and 
successful? What data did you collect? 



How are you using this information to improve the contribution of members of 
stakeholder groups in the support and counseling service activities? 

Consortia relied primarily on informal information, such as feedback at meetings or 
other events. They also tracked the level of involvement in planning, design, and 
implementation activities, reviewed the minutes of stakeholder meetings, and 
measured the degree of stakeholder involvement based on the number of 
presentations, field trips, inservices, and other forms of interaction between 
stakeholder groups and consortium and district/school staff 

Consortia considered the following areas for needed improvement: improvement of 
the linkages with post-secondary, upgrading the perceived status of Tech Prep 
programs, increased education of parents and the community regarding the value of 
technical careers and Tech Prep, and an increased level of interaction and 
involvement of business and industry. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Tech Prep System - Support Services & Counseling 
System Activity - Evaluation 



Planning 

Describe the process by which the assessment of Tech Prep- related support 
services and counseling was planned. 

Describe the strategies your consortium selected to evaluate its Tech Prep- 
related support services and counseling. 

Tech Prep consortia had included the planning for evaluating support services and 
counseling in their strategic plan as part of the overall assessment of the effectiveness 
of the program. The measures and strategies planned for assessment included 
enrollment and program participation, the high school follow-up systems, including 
the Tech Prep Identifier form, and the self-assessment. One consortium reported 
more targeted measures, such as Tech Prep course assessments, student focus 
groups, and surveys of applied curriculum instructors. 



Implementation (What was actually done?) 

Describe the strategies your consortium implemented to evaluate the 

effectiveness of its Tech Prep-related support services and counseling. 



Many consortia relied on informal information from counselors, instructors and 
staff. They used standard assessment strategies, such as the Tech Prep Identifier 
Forms, High School Follow-up, and the consortium self-assessment. One consortium 
indicated the use of the career planning survey, and the five year follow-up survey. 
Other reported using quantitative data, such as the number of students enrolled in 
Tech Prep, number of certificates awarded, number of special population students 
participating, and the numbtr of articulation agreements signed. One consortium 
had evaluated all Tech Prep courses for the past two years. 
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Continuous Improvement 

How were you able to determine if the Tech Prep-related support services and 
counseling assessment strategies you selected were appiopriate? What 
data did you collect? 

How are you using this information to improve the future assessment of Tech 
Pre-re late d support services and counseling? 

Consortia used the information gathered to further refine and guide the support anc 3 
counseling services in their member schools. 
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Minnesota Tech Prep Consortia Evaluation System 
1994/95 Cohort Self-Assessment Summary Report 



Retrospective - Support Services and Counseling 



Planning 

What processes worked best for your consortium during the planning phase of 
designing and selecting Tech Prep-related support services and 
counseling strategies? 

The processes that worked best for consortia during the planning phase for support 
services and counseling was the formation of direct communication links between 
and among different groups of stakeholders throughout the consortium. This 
included initial communication and continuous feedback about the planning from 
students, parents, counselors, staff, instructors, and the community at large. 
Stakeholders got involved in the planning process through participation on cross- 
functional teams, and participation in workshops and conferences. Several consortia 
reported that their decision to present Tech Prep as a high quality option for all 
students helped gain commitment and enhanced collaborative efforts. 



Based on your experiences, what pitfalls would you caution a new consortium 
about when planning Tech Prep-related support services and 
counseling? 



Many consortia offered the advice to build as broad a constituent base as possible from 
the start, and caution against moving ahead before consortium-wide commitment is 
reached. This requires the development of an agreed-upon vision that is supported by 
everybody. The concept of career development, in particular, needs support beyond 
the counseling community. Student support and counseling is seen as everybody’s 
job. Several consortia advised to obtain a realistic picture of the resource demands of 
Tech Prep, and to build appropriate rewards into the system for those staff who 
participate in extra assignments and tasks. Lastly, Tech Prep planning must be 
flexible and take into account local requirements and needs, especially in large 
consortia. One plan never fits all school districts. 
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Tech Prep System: Su pport Serving & Counseling 



Implementation 

When implementing Tech Prep support services and counseling, in what areas 
of activity did you experience the greatest successes? Why? 

The degree of commitment and enthusiasm among counselors and other groups at 
both the secondary and post-secondary levels was cited frequently as one of the key 
factors that contributed to the success of Tech Prep. Counselor days at the technical 
college, staff networking, Tech Prep entry and exit counseling at both the secondary 
and post-secondary levels, and initiatives such as Tech Prep portfolios and career 
education curricula attested to the successful collaboration within specific consortia. 
Other successes included the level of interest in information regarding Tech Prep. 
There was a great demand for written information, such as brochures and 
information about career pathways. Lastly, extensive inservices and continuous 
information flow contributed to the availability of accurate and up-to-date counseling 
services to students. 



Describe the three of four greatest difficulties you encountered in setting up 
support services and counseling for Tech Prep programs? 

The most frequently cited difficulty concerned the attitude of students, parents, and 
even counselors about Tech Prep as an inferior alternative to an academic education. 
This led to initially low enrollments in newly developed Tech Prep classes in some 
consortia. Secondly, the lack of adequate funding, administrative resources, and 
leadership support impeded the progress of Tech Prep in several consortia. Lastly, 
counselors often lacked appropriate labor market information to be able to advise 
students appropriately. 



Continuous Improvement 

Which areas of activity during the design and implementation of Tech Prep 
support and counseling service systems were the easiest to improve? 
Please explain why you think they were? 



The level of cooperation and communication among different levels within a given 
consortium was most easily improved in many consortia. This took the form of 
involving teachers and parents in career planning, and facilitated changes in the 
curriculum and the development of integrated courses, especially when the changes 
affected elective parts of the curriculum. Counselors were, in general, full of support 
and enthusiasm, and promoted Tech Prep in many different ways. 



So 
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Tech Prep System: Support Serv ing & fiminaeling 



Describe the areas you found most difficult to improve in Tech Prep support 
and counseling services. Please explain why you think they were. 

The most frequently mentioned area of difficulty concerned the image of Tech 
Prep and the negative attitudes against it. Parents, the community, and even the 
administrative structure of many schools still appear to favor an academic track 
over Tech Prep. A second difficulty was the lack of sufficient resources, the 
scarcity of release time for staff, and the attitude that Tech Prep was “just one 
more thing to do”. 



O 
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Section III 



Minnesota Tech Prep Consortia 
Evaluation System 

Results of Peer Brainstorming 

Session 
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MN Tech Prep Consortia Evaluation System 
Peer Brainstorming Session 
January 8, 1996 

9 aan* - 12:30 p.m., R 380 VoTech Education Building, 
University of Minnesota, St Paul Campus 



Present: 

Representing Mid-Minnesota Tech Ed Consortium: 

Dennis Schroeder 

Representing Central Minnesota Consortium: 

Steve Morgan 

Representing Oak Land Vocational Cooperative: 

Dick Young 
Ginny Karbowski 

Representing Minneapolis Public Schools: 

Joan Wilcosz 

Representing St. Paul Tech Prep: 

Nick Waldoch 

Representing Mines and Pines Tech Prep Consortium: 

Kathy Sterk 

Representing Intermediate District #287: 

Marilyn Cook 

Representing Southwest/West Central ECSU: 

Kim Lippert 

Representing Red Wing/Winona TC: 

Delores McGillivry 

Representing the MRDC: 

Jim Brown, Professor and Director of MRDC, Department of Vocational 
and Technical Education 

Dave Pucel, Professor and Head of Division of Industrial Education, 
Department of Vocational and Technical Education 
Stephan Flister, Research Assistant 
Peter Kuchinke, Research Assistant 

Representing the Minnesota Department of Children, Families, and Learning: 
Tom Ryerson, Specialist-Tech Prep/Industrial Technology 
John Sedey, Project Systems Manager 
Representing Minnesota State Colleges and Universities: 

Duane Rominger 

Not represented: 

North Country Vocational Cooperative, Bemidji 
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Summary 



Representatives from nine of the ten consortia that had completed the self- 
assessment in 1995, representatives from the Minnesota Department of Children, 
Families, and Learning, Minnesota State Colleges and Universities, and the 
Minnesota Research and Development Center for Vocational Education (MRDC) 
participated in a peer brainstorming session conducted by the MRDC, the third 
party evaluator of the Minnesota Tech Prep System. 



The session was designed to- 

• check the accuracy of the MRDC Self-Assessment Summary 
document, 

• review the status of each consortium’s Tech Prep activities with a 
peer group benchmark, 

• share successful and unsuccessful practices to inform peers and 
new consortia, and 

• provide feedback to the MRDC on the Self-Assessment Summary and 
Report processes. 



The results of the session have been incorporated into this final Self-Assessment 
Summary Report and distributed by the MRDC. 

Drs. Brown and Pucel conducted the session by discussing the seven activities 
included in each of the four Tech Prep Systems. The session was structured to 
allow participants approximately ten minutes to re-read the particular section of 
the self-assessment summary report, and then to brainstorm ideas about best 
practices and areas for improvement. 

The following is a verbatim transcript of the, ideas, issues, recommendations, and 
concerns raised by the participants and captured on wall-charts during the 
session. Comments are organized by System Activities within each Tech Prep 
System. 
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CURRICULUM AND INSTRUCTION 



Overall Planning 

1. Implementation of proposed concepts was sometimes difficult, due to lack of 
understanding of proposed items. 

2. Periodic sessions (annually?) with stakeholders would be useful to clarify 
understanding of Tech Prep. 

3. Contact persons are in danger of being given too much responsibility in 
evaluation process. 

4. Superintendents often interested only in getting their fair share of Tech Prep 
monies. 

5. Key administrators are often unaware of key goals/obligations. 

6. Principals often did not think Tech Prep money justified evaluation process, 
especially in large consortia with many districts. 

Staff Development 

1. Succeeded in sites committed to systemic change efforts (comprehensive school 
reform. 

2. Call upon external resources, including practitioners that have good credibility 
in training efforts. 

3. Staff development is a long-term, on-going process. 

4. Staff turn-over issue must be addressed. 

5. Arrange for networking opportunities of new and experienced staff. 

6. Integration topic needs groups greater than ten participants. 

7. Sell staff development idea to principals first. 

8. Arrange for job shadowing of business jobs for K-12 staff. 

Curriculum Integration 

1. Block-scheduling for students and teachers (academic and vocational in one 
block. 

2. Standards-driven model with integrated academic, SCANS, vocational 
competencies. 

3. Thematic instruction. 

4. Extensive exposure to real examples of integration in business and industry. 

5. Support attendance at national integration workshops (SREB). 

Special Populations 
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1. Special population issue confusing, some assume Tech Prep is geared solely 
toward special students. 

2. Some sites focus on LEP populations and implications for multi-language 
instruction/public relations. 

3. What changes need to be accomplish to show that Tech Prep is inclusive, for all 
students? 

4. How to balance special needs access and special needs focus? 

5. Focus on building’s counselors (special needs emphasis) participating in 
training. 

6. Questions around definition of special populations, select sites that are 
positive/inclusive. 

7. Include gifted students under definition of special population. 

Articulation 

1. Discussions between high school and post-secondary instructors clarified the 
similarities between the two. 

2. Develop an official statement of administration to support articulation. 

3. Offer joint workshops. 

4. One person needs to be responsible for articulation. 

5. Develop an overall plan for future articulation. 

6. There is an 80% match between high school and postsecondary instruction, a 
good base for articulation. 

7. The big picture needs to be clear and must include career articulation. 

8. Articulation might include the elimination of post-secondary remedial courses. 

Partnerships 

1. Partnerships among institutions in the area successful, multi-consortia 
partnerships emerging. 

2. Industry input to curricula: discussions, staff site visits, school-site visits 
(difficult to do). 

Expanded advisory board agenda, see bigger picture. 

3. Broad occupational standards approaches to key stakeholders, industry- 
standard driven. 
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4. Tech Prep has caused partnerships between secondary/postsecondary and 
secondary/secondary to emerge where none existed before (perspective building, 
curriculum sharing, etc.). 

Evaluation 

1. Career development needs to be a key focus. 

2. Knowing evaluation criteria ahead of time is helpful. 

3. Lack of information re: formal evaluation techniques, technical assistance is 
needed. 

4. Persons reporting were often very honest and provided valuable information. 

5. Staff continuity is necessary to complete the evaluation process. 

6. Need hard data, such as performance on standardized tests. 

7. Compare Tech Prep student achievement with other groups, do longitudinal 
research. 
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marketing 



Overall Planning 

1. Successful in getting secondary math and science teachers to sell Tech Prep to 
their students. 

2. Involve counselors in High School Follow-up, share information, they are now 
more willing to listen. 

3. Parents still represent an unconvinced audience. 

4. 2+2+2 is a sellable product, not a dead end. 

5. Demonstrate the multiple options of a Tech Prep track. 

6. Printed materials, such as Tech Prep catalogues, are effective and beneficial, 
especially when they show Tech Prep pathways (grades 9-12, 13-14), job 
opportunities, course, transfers, etc. 

7. Stress MNSCU transfer curricula. 

8. Assure secondary students that they are not facing extra developmental 
postsecondary courses. 

9. Develop benchmarks for all secondary students completing Tech Prep 
programs. 

Staff Development 

1. Marketing processes need to involve input and effort by all stakeholders in the 
community. 

2. Multiple and frequent messages about value of Tech Prep need to be delivered to 
have an impact. 

3. Efforts at levels higher than consortia needed (state policy, legislation). 

4. Need to work one-on-one with entire staff. 

5. Work with those who are potentially interested, never work at a single level 
exclusively. 

6. Tie discussion about Tech Prep directly to education reform. 

7. Seek district-wide support, and visibility. 

8. Use success stories to help explain program and its benefits. 

9. Provide sufficient information about Tech Prep prior to site visits. 

10. Use business testimony to present intake processes and expectations. 

11. Discuss desirability of a state- wide staff development conference. 

12. Share examples and best practices among consortia; develop showcase of good 
Tech Prep stuff. 
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13. Develop strategies to strengthen program and proceed with implementation 
during Congressional budget turmoil. 

Curriculum Integration 

1. Use of business and industry sales and technical people to show how subject 
knowledge is integrated with job duties. 

2. Use business representatives to show students the application of knowledge in 
varying settings. 

3. Emphasize performance-based curriculum integration (process/instruments). 

Special Populations 

1. Emphasize importance of marketing Tech Prep to special populations to 
counselors, encouraging them to focus on areas such as gender equity. 

2. Issue policy statement regarding equal access. 

3. Emphasize that Tech Prep is *or all students. 

4. Produce multi-language marketing materials. 

5. Stress importance of K-16 career development process for everyone. 

Articulation 

1. Tech Prep programs are not “dead end”. 

2. Important roles of certificates for articulated courses; send certificates directly 
to parents, stress savings in post-secondary tuition, fees, and time. 

3. Instructors’ enthusiasm is critical. 

4. Mail out copies of articulation agreements to inform and sell idea to 
instructors. 

5. Give administration data about the number of certificates awarded. Send 
“thank-you” notes to instructors. 

Partnerships 

1. Business representatives visit schools and speak about the value of integration. 

2. Brochures to counselors on career planning in secondary and postsecondary. 

3. Multi-consortia/district collaboration in staff development. 

4. Get employers to support Tech Prep among their employees. 

5. Match $$ with community partners re: community-wide efforts; emphasize role 
of Private Industry Councils. 
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6. Prepare to receive $$ thorough multi-sectoral partnerships. 

7. Need for state-wide school-to-work systems. 

Evaluation 

1. There is little evidence of marketing activities, too early for many consortia. 

2. Evidence of need for data comes from staff requesting more materials and 
information. 

3. Evaluation of marketing needs to be ongoing, not a one-time effort. 

4. Evaluation needs to go beyond local level and include state agencies/staff. 

5. Compare and contrast multiple co mm unities* efforts and outcomes. 
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STUDENT ASSESS 






[* AND EVALUATION 



Overall Planning 

1. Joint K-12 and Community/Technical College staff planning efforts are 
desirable. 

2. Emphasis on moving to authentic assessment. 

3. There is a need for instructors to have authority to identify 
goals/criteria/performance levels and to determine student levels. 

4. A policy decision is needed whether to focus on vocational or integrated 
academic curricula. 

5. Offering of developmental courses at the post-secondary level without 
application to Tech Prep programs; strong academic foundations are required of 
everyone. 



Staff Development 

1. Authentic assessment is emerging as an emphasis. 

2. Student assessment is one of the weakest components of the curriculum, this 
became apparent with the Graduation Rule. 

3. Educators tend to assume that assessment focuses on “in-class”, not on career 
outcomes in the ‘real world’. 

4. Consortia are attempting to integrate authentic assessment with Graduation 
Rule efforts (requires low level articulation.. 

Curriculum Integration 

1. Performance assessment 

2. Exploring applying ASSET test to see what remediation is needed. This can 
have important implications on needed services. 

3. Need State Committee discussion and decision over use of ASSET or ASAP. 

Special Populations 

1. Concern about Graduation Rule meetings regarding: What students leave 
school with/without. 
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Articulation 



no comments 
Partnerships 

1. Too early in some Tech Prep programs to be relevant. 

2. Periodic meetings to discuss students’ school-to-work performance with 
business and industry reps, supervisors, and students 

3. Considering getting employers’ entrance exams to see how students perform on 
exams. 

Evaluation 
no comments 
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SUPPORT SERVICES AND COUNSELING 



Overall Planning 

1. Be sure to include support services staff in planning process. 

2. Clarify and communicate various staff groups’ responsibilities. 

3. Counselors are often caught in the middle. 

Staff Development 

1. Change counselors’ mindset regarding occupational opportunities for Tech 
Prep graduates, encourage them to spread the word. 

2. Broad base of target population very important. 

3. Content must be real world relevant. 

4. Give staff a clear understanding of Tech Prep concepts prior to their 
interactions with business personnel and vice versa. 

5. Orientation to training recipients and training providers. 

6. Use time efficiently, pulling staff from classrooms is difficult. 

7. Building-level activities seem to be more appropriate than district- wide 
activities. 

Curriculum Integration 

no comments 

Special Populations 

1. Keep staff well-informed. 

2. Communicate types/eligibilities for services available. 

3. Clarify roles of case managers for special Ed students. 

Curriculum Integration 

1. Support services personnel are consistently involved in Tech Prep processes, 
especially in curriculum integration. 

2. Broad approach to school-to-work transition processes needed (beyond special 
education). 
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Articulation 

1. Support staff must be knowledgeable of articulated courses. 

Partnerships 

Ensure presence of critical members in group meetings. 

Evaluation 

1. Not a high priority yet. 

2. Field trips provide informal feedback re: Postsecondary programs. 
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The following represents the results of a final feed-back segment at the end of the 
session. Participants were asked for overall comments regarding the assessment 
and Tech Prep. 

1. The assessment process forces the consortium to focus on areas that might 

have otherwise gotten overlooked. 

2. The summary report has high face validity, consortium representatives find 
their own experience accurately reflected in the report. 

3. There is a need to define Tech Prep within the larger context of education 

reform on the state and national level. 

4. There is a need to determine the benefits of Tech Prep to the overall education 

system in Minnesota. 

5. Staff development is a critical issues, and needs to occur ongoing and 
continuously, especially in light of staff turnover. 

6. Involvement by school principals is critical. 

7. There is growing evidence of partnerships among consortia, the formation of 
“superconsortia”. 

8. There is a need for a state-wide vision for Tech Prep, such as the Florida 

example. 

9. There is a need for longitudinal research on the labor market experience of 
Tech Prep graduates. 

10. Marketing for Tech Prep needs to also occur system-wide, possibly through 
state- wide showcases of best practices at a Tech Prep fair. 
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